The third example deals with finding ICA filters for natural images and, based on the ICA decomposition, removing noise from images corrupted with additive Gaussian noise.
A set of digitized natural images were used. Denote the vector of pixel gray levels in an image window by . Note that, contrary to the other two applications in the previous sections, we are not this time considering multivalued time series or images changing with time; instead the elements of are indexed by the location in the image window or patch. The sample windows were taken at random locations. The 2D structure of the windows is of no significance here: row by row scanning was used to turn a square image window into a vector of pixel values. The independent components of such image windows are represented in Fig. 4. Each window in this Figure corresponds to one of the columns of the mixing matrix . Thus an observed image window is a superposition of these windows as in (5), with independent coefficients.
Now, suppose a noisy image model holds:
We have recently introduced another, statistically principled method called Sparse Code Shrinkage [22]. It is very closely related to independent component analysis. Briefly, if we model the density of by ICA, and assume Gaussian, then the Maximum Likelihood (ML) solution for given the measurement can be developed in the signal model (48).
The ML solution can be simply computed, albeit approximately, by using
a decomposition that is an orthogonalized version of ICA.
The transform is given by
(46) 
It was shown in [22] that, assuming a Laplacian density for s_{i}, the ML solution for s_{i} is given by a ``shrinkage function'' , or in vector form, . Function g(.) has a characteristic shape: it is zero close to the origin and then linear after a cutting value depending on the parameters of the Laplacian density and the Gaussian noise density. Assuming other forms for the densities, other optimal shrinkage functions can be derived [22].
In the Sparse Code Shrinkage method, the shrinkage operation is
performed in the rotated space, after which the estimate for the
signal in the original space is given by rotating back:
(47) 
The rotation operator is such that the sparsity of the components is maximized. This operator can be learned with a modification of the FastICA algorithm; see [22] for details.
A noise cleaning result is shown in Fig. 15. A noiseless image and a noisy version, in which the noise level is 50 % of the signal level, are shown. The results of the Sparse Code Shrinkage method and classic wiener filtering are given, indicating that Sparse Code Shrinkage may be a promising approach. The noise is reduced without blurring edges or other sharp features as much as in wiener filtering. This is largely due to the strongly nonlinear nature of the shrinkage operator, that is optimally adapted to the inherent statistics of natural images.
