We propose in this paper a probabilistic approach for adaptive inference
of generalized nonlinear classification that combines the computational
advantage of a parametric solution with the flexibility of sequential sampling
techniques. We regard the parameters of the classifier as latent
states in a first order Markov process and propose an algorithm which
can be regarded as variational generalization of standard Kalman filtering.
The variational Kalman filter is based on two novel lower bounds
that enable us to use a non-degenerate distribution over the adaptation
rate. An extensive empirical evaluation demonstrates that the proposed
method is capable of infering competitive classifiers both in stationary
and non-stationary environments. Although we focus on classification,
the algorithm is easily extended to other generalized nonlinear models.