Ambiguities of ICA

In the ICA model in Eq. (4), it is easy to see that the following ambiguities will hold:

1. We cannot determine the variances (energies) of the independent components.

The reason is that, both ${\bf s}$ and ${\bf A}$ being unknown, any scalar multiplier in one of the sources s_i could always be cancelled by dividing the corresponding column ${\bf a}_i$ of ${\bf A}$ by the same scalar; see eq. (5). As a consequence, we may quite as well fix the magnitudes of the independent components; as they are random variables, the most natural way to do this is to assume that each has unit variance: $E\{s_i^2\} = 1$. Then the matrix ${\bf A}$ will be adapted in the ICA solution methods to take into account this restriction. Note that this still leaves the ambiguity of the sign: we could multiply the an independent component by -1 without affecting the model. This ambiguity is, fortunately, insignificant in most applications.

2. We cannot determine the order of the independent components.

The reason is that, again both ${\bf s}$ and ${\bf A}$ being unknown, we can freely change the order of the terms in the sum in (5), and call any of the independent components the first one. Formally, a permutation matrix ${\bf P}$ and its inverse can be substituted in the model to give ${\bf x}= {\bf A}{\bf P}^{-1} {\bf P}{\bf s}$. The elements of ${\bf P}{\bf s}$ are the original independent variables s_j, but in another order. The matrix ${\bf A} {\bf P}^{-1}$ is just a new unknown mixing matrix, to be solved by the ICA algorithms.