We present a framework for the classification of visual
processes that are best modeled with spatio-temporal autoregressive
models. The new framework combines the
modeling power of a family of models known as dynamic
textures and the generalization guarantees, for classification,
of the support vector machine classifier. This combination
is achieved by the derivation of a new probabilistic
kernel based on the Kullback-Leibler divergence (KL) between
Gauss-Markov processes. In particular, we derive the
KL-kernel for dynamic textures in both 1) the image space,
which describes both the motion and appearance components
of the spatio-temporal process, and 2) the hidden state
space, which describes the temporal component alone. Together,
the two kernels cover a large variety of video classification
problems, including the cases where classes can
differ in both appearance and motion and the cases where
appearance is similar for all classes and only motion is
discriminant. Experimental evaluation on two databases
shows that the new classifier achievessuperior performance
over existing solutions.