Uncorrelated variables are only partly independent

A weaker form of independence is uncorrelatedness. Two random variables ${\bf y}_1$ and y₂ are said to be uncorrelated, if their covariance is zero:

$$E\{y_1 y_2\}-E\{y_1\}E\{y_2\}=0$$ (10)

If the variables are independent, they are uncorrelated, which follows directly from Eq. (11), taking h₁(y₁)=y₁ and h₂(y₂)=y₂.

On the other hand, uncorrelatedness does not imply independence. For example, assume that (y₁,y₂) are discrete valued and follow such a distribution that the pair are with probability 1/4 equal to any of the following values: (0,1),(0,-1),(1,0),(-1,0). Then y₁ and y₂ are uncorrelated, as can be simply calculated. On the other hand,

$$E\{ y_1^2 y_2^2\}=0 \neq 1/4= E\{y_1^2\}E\{y_2^2\}.$$ (11)

so the condition in Eq. (11) is violated, and the variables cannot be independent.

Since independence implies uncorrelatedness, many ICA methods constrain the estimation procedure so that it always gives uncorrelated estimates of the independent components. This reduces the number of free parameters, and simplifies the problem.