next up previous
Next: Principles of ICA estimation Up: What is independence? Previous: Uncorrelated variables are only

   
Why Gaussian variables are forbidden

The fundamental restriction in ICA is that the independent components must be nongaussian for ICA to be possible.

To see why gaussian variables make ICA impossible, assume that the mixing matrix is orthogonal and the si are gaussian. Then x1 and x2 are gaussian, uncorrelated, and of unit variance. Their joint density is given by

\begin{displaymath}p(x_1,x_2)=\frac{1}{2\pi}\exp(-\frac{x_1^2+x_2^2}{2})
\end{displaymath} (12)

This distribution is illustrated in Fig. 7. The Figure shows that the density is completely symmetric. Therefore, it does not contain any information on the directions of the columns of the mixing matrix ${\bf A}$. This is why ${\bf A}$ cannot be estimated.


  
Figure 7: The multivariate distribution of two independent gaussian variables.
\resizebox{.50\textwidth}{!}{
\includegraphics{gaussian}}

More rigorously, one can prove that the distribution of any orthogonal transformation of the gaussian (x1,x2) has exactly the same distribution as (x1,x2), and that x1 and x2 are independent. Thus, in the case of gaussian variables, we can only estimate the ICA model up to an orthogonal transformation. In other words, the matrix ${\bf A}$ is not identifiable for gaussian independent components. (Actually, if just one of the independent components is gaussian, the ICA model can still be estimated.)


next up previous
Next: Principles of ICA estimation Up: What is independence? Previous: Uncorrelated variables are only
Aapo Hyvarinen
2000-04-19