redirect.html


This page has moved to www.cs.helsinki.fi/aapo.hyvarinen/pub.html.
Your browser should automatically take you there.






Selected on-line publications by Aapo Hyvärinen

(as of April 2003)

(A complete publication list also is available in pdf and PostScript.)

marks papers that have been put on this page during the year 2002

marks the most important publications

Natural image statistics and the visual cortex, or extensions of ICA

The FastICA algorithm

Independent component analysis and blind source separation: further papers

Independent Component Analysis: The Book

The papers below are all connected to ICA, so you may first want to know what is independent component analysis.


Natural image statistics and the visual cortex, or extensions of ICA

Research group home page

J. Hurri and A. Hyvärinen. Simple-Cell-Like Receptive Fields Maximize Temporal Coherence in Natural Video. Neural Computation, 15(3):663-691, 2003.
Postscript  gzipped PostScript   pdf
[Analyzes the temporal correlations of natural image sequences, and shows that temporal coherence (correlation) leads to emergence of the same kind of receptive fields that have previously been found by ICA (sparse coding).]

P.O. Hoyer and A. Hyvärinen. A Multi-Layer Sparse Coding Network Learns Contour Coding from Natural Images. Vision Research, 42(12):1593-1605, 2002.
Abstract  Postscript  gzipped PostScript   pdf
[Analyzes the independent components of complex cell outputs, and shows that they are higher-order contour coding units, or sometimes end-stopped complex cells.]

A. Hyvärinen and P. O. Hoyer. A Two-Layer Sparse Coding Model Learns Simple and Complex Cell Receptive Fields and Topography from Natural Images. Vision Research, 41(18):2413-2423, 2001.
Abstract  Postscript  gzipped PostScript  pdf
[Uses a simplified version of topographic ICA (see below) to model the topography (spatial organization) of the cells in primary visual cortex, in addition to simple cell and complex cell receptive fields.]

A. Hyvärinen, P.O. Hoyer and M. Inki. Topographic Independent Component Analysis. Neural Computation, 13(7):1525-1558, 2001.
Abstract  Postscript  gzipped PostScript   pdf
[Introduces an extension of ICA. The dependencies of the estimated "independent" components are visualized as a topographic order. A new principle for topographic organization, based on higher-order statistics. Applied on image data, both topography and complex cell properties emerge (see also above). This paper is more concentrated on the mathematical formulation, and the preceding one on the biological application.]

A. Hyvärinen and P.O. Hoyer. Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces. Neural Computation, 12(7):1705-1720, 2000.
Abstract  Postscript  gzipped PostScript  pdf
[A simpler extension of ordinary ICA which was a precursor of topographic ICA. Here the goal of independence of scalar independent components is replaced by the independence of the norms of projections on certain subspaces. This is applied on image data, and complex cell properties are shown to emerge.]

A. Hyvärinen and P. O. Hoyer. Emergence of Topography and Complex Cell Properties from Natural Images using Extensions of ICA. Advances in Neural Information Processing Systems 12 (NIPS*99), pp. 827-833, 2000.
Postscript  gzipped PostScript.
[A short summary of the three preceding papers.]

P.O. Hoyer and A. Hyvärinen. Independent Component Analysis Applied to Feature Extraction from Colour and Stereo Images. Network: Computation in Neural Systems, 11(3):191-210, 2000.
Postscript  gzipped PostScript  pdf
[Shows that the independent components of colour and stereo images are quite similar to the corresponding V1 receptive fields.]

A. Hyvärinen. An alternative approach to infomax and independent component analysis . Neurocomputing, 44-46(C):1089-1097.
Postscript  gzipped PostScript
[Considers the problem of nonrobustness of the ordinary infomax approach to ICA, and proposes a solution to this problem, based on a more biological noise model.]

MATLAB code for estimating ICA, ISA, and TICA bases from image data is also available (by P. O. Hoyer).


The FastICA algorithm

A MATLAB package for the FastICA algorithm is also available.

Project home page

A. Hyvärinen. Fast and Robust Fixed-Point Algorithms for Independent Component Analysis. IEEE Transactions on Neural Networks 10(3):626-634, 1999.
Abstract  Postscript  gzipped PostScript  html pdf
[The fundamental paper on the FastICA algorithm, which is a computationally very efficient method for performing ICA. This article is based on the TechRep "ICA by Minimization of Mutual Information".]

A. Hyvärinen and E. Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks, 13(4-5):411-430, 2000.
html (and Japanese version)  Postscript  gzipped PostScript   pdf
[A tutorial text on ICA in general, and FastICA in particular.]

A. Hyvärinen and E. Oja. A Fast Fixed-Point Algorithm for Independent Component Analysis. Neural Computation, 9(7):1483-1492, 1997.
Abstract  Postscript  gzipped PostScript.
[Introduced the original, cumulant-based, form of the FastICA algorithm.]

 A. Hyvärinen. New Approximations of Differential Entropy for Independent Component Analysis and Projection Pursuit.  In Advances in Neural Information Processing Systems 10 (NIPS*97), pp. 273-279, MIT Press, 1998.
Abstract  Postscript  gzipped PostScript   pdf. Related TechRep: Postscript  gzipped PostScript.
[Introduced approximations of differential entropy used in the derivation of the FastICA algorithm in the IEEE Transactions paper above.]

E. Bingham and A. Hyvärinen A fast fixed-point algorithm for independent component analysis of complex-valued signals. Int. J. of Neural Systems, 10(1):1-8, 2000.
Postscript  gzipped PostScript  Matlab code
[A version of FastICA for complex-valued data.]

A. Hyvärinen.  One-Unit Contrast Functions for Independent Component Analysis: A Statistical Analysis.  In Neural Networks for Signal Processing VII (Proc. IEEE NNSP Workshop '97, Amelia Island, Florida), pp. 388--397, 1997.
Abstract  Postscript  gzipped PostScript.
[Gives a statistical analysis of the maximum nongaussianity framework used in FastICA and some other ICA algorithms given below.]

A. Hyvärinen. The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis. Neural Processing Letters, 10(1):1-5.
Abstract  Postscript  gzipped PostScript.
[Shows how the FastICA algorithms can be interpreted as maximum likelihood estimation.]


Independent Component Analysis and Blind Source Separation: Further Papers

Noisy ICA with applications to image denoising

Project home page

A. Hyvärinen. Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation. Neural Computation, 11(7):1739--1768, 1999.
Abstract  Postscript  gzipped PostScript
[Describes sparse code shrinkage, which is a new method for denoising of images etc. It is a kind of a combination of independent component analysis and wavelet shrinkage ideas.]

A. Hyvärinen, P. Hoyer and E. Oja. Image Denoising by Sparse Code Shrinkage. In S. Haykin and B. Kosko (eds), Intelligent Signal Processing, IEEE Press, 2001.
Postscript  gzipped PostScript
[Describes sparse code shrinkage in much more detail.]

A. Hyvärinen. Independent Component Analysis in the Presence of Gaussian Noise by Maximizing Joint Likelihood. Neurocomputing, 22:49-67, 1998.
Abstract  Postscript  gzipped PostScript.
[Older work on noisy independent component analysis and its connections to competitive learning.]

A. Hyvärinen. Gaussian Moments for Noisy Independent Component Analysis.
IEEE Signal Processing Letters, 6(6):145--147, 1999.
Abstract  Postscript  gzipped PostScript. Longer paper with proofs (Proc. ISCAS'99): gzipped PostScript.
[Shows how to modify the FastICA algorithm obtain consistent estimators when the data is corrupted by Gaussian noise. Introduces the concept of Gaussian moments.]

ICA with overcomplete bases

A. Hyvärinen and M. Inki. Estimating overcomplete independent component bases for image windows. Journal of Mathematical Imaging and Vision, 17:139-152, 2002.
Abstract  Postscript  gzipped PostScript  pdf
[Discusses different methods for overcomplete basis estimation from images, and proposes two new, computationally efficient algorithms.]

A. Hyvärinen, R. Cristescu and E. Oja. A fast algorithm for estimating overcomplete ICA bases for image windows. In Proc. Int. Joint Conf on Neural Networks, Washington D.C., 1999.
Abstract  Postscript  gzipped PostScript
[A version of FastICA that estimates overcomplete bases, especially suitable for image data.]

Finding components using temporal structure

A. Hyvärinen. Complexity Pursuit: Separating interesting components from time-series. Neural Computation, 13(4):883--898, 2001.
Abstract  Postscript  gzipped PostScript
[Introduces the concept of complexity pursuit, which means finding projections of time series (signals) that have minimum complexity. Also introduces simple approximations of complexity that take into account both nongaussianity and autocorrelations.]

A. Hyvärinen. Blind source separation by nonstationarity of variance: A cumulant-based approach. IEEE Trans. on Neural Networks, 12(6):1471-1474, 2001.
Abstract  Postscript  gzipped PostScript   pdf
[Formulates the less-known separation criterion on variance nonstationarity using cumulants, and proposes a fast fixed-point algorithm.]

A. Hyvärinen. Independent Component Analysis for Time-dependent Stochastic Processes.  In Proc. Int. Conf. on Artificial Neural Networks (ICANN'98), Skövde, Sweden, pp. 541-546, 1998.
Abstract  Postscript   gzipped PostScript
[This paper shows that in ICA, it is often useful to preprocess the data by computing the innovation processes.]

Neural learning rules for ICA

A. Hyvärinen and E. Oja. Independent Component Analysis by General Non-linear Hebbian-like Learning Rules.  Signal Processing,  64(3):301-313, 1998.
Abstract  Postscript  gzipped PostScript .
[Introduces one-unit adaptive algorithms related to FastICA. Shows how to estimate a coefficient that allows the estimation of both sub- and super-Gaussian independent component using a single nonlinearity.]

A. Hyvärinen.  One-Unit Learning Rules for Independent Component Analysis.
In Advances in Neural Information Processing Systems 9 (NIPS*96), MIT Press, pp.  480--486, 1997.
Abstract  Postscript  gzipped PostScript.
[A nice if somewhat out-dated overview on the one-unit ICA algorithms, including fixed-point and adaptive versions.]

A. Hyvärinen and E. Oja. Simple Neuron Models for Independent Component Analysis. Int. Journal of Neural Systems, 7(6):671-687, 1996.
Postscript  gzipped PostScript.
[Some older work, rather out-dated.]

Nonlinear ICA

Project home page

A. Hyvärinen and P. Pajunen. Nonlinear Independent Component Analysis: Existence and Uniqueness results. Neural Networks 12(3): 429--439, 1999.
Abstract  Postscript  gzipped PostScript
[Shows that the solution of the nonlinear ICA problem is highly non-unique, and proposes a restriction of the model that does make the solution unique.]

P. Pajunen, A. Hyvärinen and J. Karhunen. Non-Linear Blind Source Separation by Self-Organizing Maps. In Proc. Int. Conf. on Neural Information Processing, Hong Kong, pp. 1207-1210, 1996.
Postscript  gzipped PostScript.
[Describes a simple if limited method for doing nonlinear (nonparametric) ICA for subgaussian signals.]

Overlearning and priors in ICA

A. Hyvärinen and R. Karthikesh. Imposing sparsity on the mixing matrix in independent component analysis. Neurocomputing, 49:151-162, 2002 (Special Issue on ICA and BSS).
Abstract  Postscript  gzipped PostScript
[Shows how to implement the prior information on the sparsity of the mixing matrix in a very simple way as conjugate priors.]

A. Hyvärinen, J. Särelä and R. Vigário. Bumps and Spikes: Artifacts Generated by Independent Component Analysis with Insufficient Sample Size. In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA'99), pp. 425-429, Aussois, France, 1999.
Abstract  Postscript (a very large file)  gzipped PostScript.
[Describes the phenomenon of overlearning in ICA.]

Miscellaneous

A. Hyvärinen and E. Bingham. Connection between multi-layer perceptrons and regression using independent component analysis. Neurocomputing, 50(C):211-222, 2003.
Abstract  Postscript  gzipped PostScript
[Shows that MLP's can be interpreted as estimating an ICA model for the data, and doing regression using that model.]

J. Himberg and A. Hyvärinen. Independent component analysis for binary data: An experimental study . In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA2001), San Diego, California, 2001.
Postscript  gzipped PostScript
[Shows how to do ICA on binary data using ordinary FastICA.]

Reviews

A. Hyvärinen and E. Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks, 13(4-5):411-430, 2000.
html (and Japanese version)  Postscript  gzipped PostScript   pdf
[A tutorial text on ICA in general, and FastICA in particular (already mentioned above.)]

A. Hyvärinen, J. Karhunen and E. Oja. Introductory Chapter of the book Independent Component Analysis
Postscript   pdf
[A short very accessible introduction to ICA (and our book).]

A. Hyvärinen. Survey on Independent Component Analysis. Neural Computing Surveys 2:94--128, 1999.
html  Postscript   gzipped PostScript   pdf
[A comprehensive review on ICA estimation methods.]

A. Hyvärinen and Y. Kano. Independent component analysis for non-normal factor analysis. Proc. International Meeting of the Psychometric Society (IMPS2001), Osaka, Japan.
Postscript  pdf
[Tutorial of ICA for people who already are familiar with factor analysis.]


Last modified in Apr 2003 by Aapo Hyvärinen Mail to maintainer: Aapo.Hyvarinen@hut.fi