In the ICA model in Eq. (4), it is easy to see that the following ambiguities will hold:
1. We cannot determine the variances (energies) of the independent components.
The reason is that, both
and
being unknown, any scalar
multiplier in one of the sources si could always be cancelled by
dividing the corresponding column
of
by the same scalar;
see eq. (5). As a consequence, we may quite as well fix the
magnitudes of the independent components; as they are random
variables, the most natural way to do this is to assume that each has
unit variance:
.
Then the matrix
will be adapted in
the ICA solution methods to take into account this restriction.
Note that this still leaves the ambiguity of the sign: we could
multiply the an independent component by -1 without affecting the model. This
ambiguity is, fortunately, insignificant in most applications.
2. We cannot determine the order of the independent components.
The reason is that, again both
and
being unknown, we can
freely change the order of the terms in the sum in (5), and call
any of the independent components the first one. Formally, a
permutation matrix
and its inverse can be substituted in the model to
give
.
The elements of
are the original
independent variables sj, but in another order. The matrix
is just a new unknown mixing matrix, to be solved by the ICA
algorithms.