Why Gaussian variables are forbidden

The fundamental restriction in ICA is that the independent components must be nongaussian for ICA to be possible.

To see why gaussian variables make ICA impossible, assume that the mixing matrix is orthogonal and the s_i are gaussian. Then x₁ and x₂ are gaussian, uncorrelated, and of unit variance. Their joint density is given by

$$p(x_1,x_2)=\frac{1}{2\pi}\exp(-\frac{x_1^2+x_2^2}{2})$$ (12)

This distribution is illustrated in Fig. 7. The Figure shows that the density is completely symmetric. Therefore, it does not contain any information on the directions of the columns of the mixing matrix ${\bf A}$. This is why ${\bf A}$ cannot be estimated.

  

Figure 7: The multivariate distribution of two independent gaussian variables.

More rigorously, one can prove that the distribution of any orthogonal transformation of the gaussian (x₁,x₂) has exactly the same distribution as (x₁,x₂), and that x₁ and x₂ are independent. Thus, in the case of gaussian variables, we can only estimate the ICA model up to an orthogonal transformation. In other words, the matrix ${\bf A}$ is not identifiable for gaussian independent components. (Actually, if just one of the independent components is gaussian, the ICA model can still be estimated.)