Performance in the exponential power family

Now we shall treat the question of choosing the contrast function Gin practice. It is useful to analyze the implications of the theoretical results of the preceding section by considering the following exponential power family of density functions:

$$f_{\alpha}(s)=k_1\exp(k_2 \vert s\vert^{\alpha})$$ (11)

where ${\alpha}$ is a positive parameter, and k₁,k₂ are normalization constants that ensure that $f_{\alpha}$ is a probability density of unit variance. For different values of alpha, the densities in this family exhibit different shapes. For $0<{\alpha}<2$, one obtains a sparse, super-Gaussian density (i.e., a density of positive kurtosis). For ${\alpha}=2$, one obtains the Gaussian distribution, and for ${\alpha}>2$, a sub-Gaussian density (i.e., a density of negative kurtosis). Thus the densities in this family can be used as examples of different non-Gaussian densities.

Using Theorem 2, one sees that in terms of asymptotic variance, an optimal contrast function for estimating an independent component whose density function equals $f_{\alpha}$, is of the form:

$$G_{opt}(u)=\vert u\vert^{\alpha}$$ (12)

where the arbitrary constants have been dropped for simplicity. This implies roughly that for super-Gaussian (resp. sub-Gaussian) densities, the optimal contrast function is a function that grows slower than quadratically (resp. faster than quadratically). Next, recall from Section 3.1.3 that if G(u) grows fast with |u|, the estimator becomes highly non-robust against outliers. Taking also into account the fact that most independent components encountered in practice are super-Gaussian [3,25], one reaches the conclusion that as a general-purpose contrast function, one should choose a function G that resembles rather

$$G_{opt}(u)=\vert u\vert^{\alpha}, \mbox{where } {\alpha}<2.$$ (13)

The problem with such contrast functions is, however, that they are not differentiable at 0 for $\alpha\leq 1$. Thus it is better to use approximating differentiable functions that have the same kind of qualitative behavior. Considering $\alpha=1$, in which case one has a double exponential density, one could use instead the function $G_1(u)=\log\cosh a_1 u$where $a_1\geq 1$ is a constant. Note that the derivative of G₁ is then the familiar tanh function (for a₁=1). In the case of $\alpha<1$, i.e., highly super-Gaussian independent components, one could approximate the behavior of G_opt for large u using a Gaussian function (with a minus sign): $G_2(u)=-\exp(-a_2 u^2/2)$, where a₂ is a constant. The derivative of this function is like a sigmoid for small values, but goes to 0 for larger values. Note that this function also fulfills the condition in Theorem 3, thus providing an estimator that is as robust as possible in the framework of estimators of type (8). As regards the constants, we have found experimentally $1\leq a_1\leq 2$ and a₂=1 to provide good approximations.