## Nonlinear state-space models

In many cases, measurements originate from a dynamical system and form
time series. In such cases, it is often useful to model the dynamics
in addition to the instantaneous observations. We have extended the
nonlinear factor analysis model by adding a nonlinear model for the
dynamics of the sources in (Valpola and Karhunen, 2002). This results in a state-space model where
the sources can be interpreted as the internal state of the underlying
generative process. The variational Bayesian method developed in (Valpola and Karhunen, 2002) is called
nonlinear dynamic factor analysis (NDFA).
An important advantage of the NDFA method in (Valpola and Karhunen, 2002) is its ability to learn
a high-dimensional latent source space. We have also reasonably solved
computational and over-fitting problems which have been major
obstacles in developing this kind of unsupervised methods thus far.
Potential applications of the NDFA method include prediction and process
monitoring, control and identification. A drawback of the original NDFA
method is that it is computationally quite demanding. On the other hand,
it often provides very good results.

## Detection of process state changes

One potential application for the nonlinear state-space model and the
NDFA method introduced in (Valpola and Karhunen, 2002) is process monitoring. In (Ilin et al., 2004), we have
shown that the NDFA method can learn a model which is capable of
detecting an abrupt change in the underlying dynamics of a fairly
complex nonlinear process. The NDFA method performs in this application
clearly better than the compared standard approaches (Ilin et al., 2004).

## Stochastic nonlinear model-predictive control

For controlling a dynamical system, control inputs are added to the
nonlinear state-space model (NDFA) introduced in (Valpola and Karhunen, 2002). In (Raiko and Tornio, 2005), we study
three different control schemes in this setting. Direct control is
based on using the internal forward model directly. It is fast to use,
but requires the learning of a policy mapping, which is hard to do
well. Optimistic inference control is a novel method based on Bayesian
inference answering the question: "Assuming success in the end, what
will happen in near future?" It is based on a single probabilistic
inference but unfortunately neither of the two tested inference
algorithms works well with it. The third control scheme is stochastic
nonlinear model-predictive control, which is based on optimizing
control signals based on maximising a utility function.

Simulation resultss with a cart-pole swing-up task in (Raiko and Tornio, 2005) confirm that
selecting actions based on a state-space model instead of the observation
directly has many benefits: First, it is more resistant to noise because it
implicitly involves filtering. Second, the observations (without
history) do not always carry enough information about the system
state. Third, when nonlinear dynamics are modelled by a function
approximator such as an multilayer perceptron network, a state-space
model can find such a representation of the state that it is more
suitable for the approximation and thus more predictable (Raiko and Tornio, 2005).

The animation shows a successfull swingup performed by nonlinear
model-predictive control using a prediction horizon of 40 time
steps. The top subfigure shows the cart-pole system at current time in
black and predictions in grey. The middle figure shows the development
of hidden states and their predictions such that the current time is
marked by the vertical dashed line. The bottom figure shows the place
and speed of the cart, angle and angle speed of the pole, and the
force applied to the cart.

### References

A. Ilin, H. Valpola, and E. Oja, "Nonlinear dynamical factor analysis for
state change detection". *IEEE Trans, on Neural Networks*, vol. 15,
no. 3, 2004, pp. 559-575. Pdf (766k).

T. Raiko and M. Tornio, "Learning nonlinear state-space models for control".
In *Proc. Int. Joint Conf. on Neural Networks (IJCNN'05)*,
Montreal, Canada, July 2005, pp. 815-820.
Pdf (192k).

H. Valpola and J. Karhunen, "An unsupervised ensemble learning method for
nonlinear dynamic state-space models". *Neural Computation*, vol. 14,
no. 11, 2002, pp. 2647-2692.Pdf
(937k).

Page maintained by juha.karhunen at hut.fi,
last updated Wednesday, 01-Oct-2008 10:30:18 EEST