next up previous
Next: The Infomax Principle Up: Maximum Likelihood Estimation Previous: Maximum Likelihood Estimation

The likelihood

A very popular approach for estimating the ICA model is maximum likelihood estimation, which is closely connected to the infomax principle. Here we discuss this approach, and show that it is essentially equivalent to minimization of mutual information.

It is possible to formulate directly the likelihood in the noise-free ICA model, which was done in [38], and then estimate the model by a maximum likelihood method. Denoting by ${\bf W}=({\bf w}_1,...,{\bf w}_n)^T$ the matrix ${\bf A}^{-1}$, the log-likelihood takes the form [38]:

L=\sum_{t=1}^T \sum_{i=1}^n \log f_i({\bf w}_i^T {\bf x}(t))+T\log\vert\det {\bf W}\vert
\end{displaymath} (27)

where the fi are the density functions of the si (here assumed to be known), and the ${\bf x}(t),t=1,...,T$ are the realizations of ${\bf x}$. The term $\log\vert\det {\bf W}\vert$ in the likelihood comes from the classic rule for (linearly) transforming random variables and their densities [36]: In general, for any random vector ${\bf x}$ with density px and for any matrix ${\bf W}$, the density of ${\bf y}={\bf W}{\bf x}$ is given by $p_x({\bf W}{\bf x})\vert\det{\bf W}\vert$.

Aapo Hyvarinen