(A complete publication list also is available in pdf and PostScript.)
marks papers that have been put on this page during the year 2002
marks the most important publications
The papers below are all connected to ICA, so you may first want to know what is independent component analysis.
J. Hurri and A. Hyvärinen. Simple-Cell-Like Receptive Fields Maximize Temporal Coherence in Natural Video.
Neural Computation, 15(3):663-691, 2003.
Postscript gzipped PostScript
pdf
[Analyzes the temporal correlations of natural image sequences, and shows that temporal coherence (correlation) leads to emergence of the same kind of receptive fields that have previously been found by ICA (sparse coding).]
P.O. Hoyer and A. Hyvärinen. A Multi-Layer Sparse Coding Network Learns Contour Coding from Natural Images.
Vision Research, 42(12):1593-1605, 2002.
Abstract
Postscript gzipped
PostScript pdf
[Analyzes the independent components of complex cell outputs, and shows that they are higher-order contour coding units, or sometimes end-stopped complex cells.]
A. Hyvärinen and P. O. Hoyer.
A Two-Layer Sparse Coding Model Learns Simple and Complex
Cell Receptive Fields and Topography from Natural Images.
Vision Research, 41(18):2413-2423, 2001.
Abstract
Postscript gzipped
PostScript pdf
[Uses a simplified version of topographic ICA (see below) to model the topography (spatial organization) of the cells in primary visual cortex, in addition to simple cell and complex cell receptive fields.]
A. Hyvärinen, P.O. Hoyer and M. Inki. Topographic Independent
Component Analysis. Neural Computation, 13(7):1525-1558, 2001.
Abstract
Postscript gzipped
PostScript pdf
[Introduces an extension of ICA. The dependencies of the
estimated "independent" components are visualized as a topographic
order. A new principle for topographic organization, based on
higher-order statistics. Applied on image data, both topography and
complex cell properties emerge (see also above). This paper is more
concentrated on the mathematical formulation, and the preceding one on
the biological application.]
A. Hyvärinen and P.O. Hoyer. Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces.
Neural Computation, 12(7):1705-1720, 2000.
Abstract
Postscript gzipped
PostScript pdf
[A simpler extension of ordinary ICA which was a precursor of topographic ICA. Here the goal of independence of scalar independent components is replaced by the independence of the norms of projections on certain subspaces. This is applied on image data, and complex cell properties are shown to emerge.]
A. Hyvärinen and P. O. Hoyer.
Emergence of Topography and Complex Cell Properties
from Natural Images using Extensions of ICA.
Advances in Neural Information Processing Systems 12 (NIPS*99), pp. 827-833, 2000.
Postscript
gzipped PostScript.
[A short summary of the three preceding papers.]
P.O. Hoyer and A. Hyvärinen. Independent Component Analysis Applied to Feature Extraction from Colour and Stereo Images.
Network: Computation in Neural Systems, 11(3):191-210, 2000.
Postscript gzipped
PostScript pdf
[Shows that the independent components of colour and stereo images are quite similar to the corresponding V1 receptive fields.]
A. Hyvärinen.
An alternative approach to infomax and independent component analysis
. Neurocomputing, 44-46(C):1089-1097.
Postscript gzipped
PostScript
[Considers the problem of nonrobustness of the ordinary infomax approach to ICA, and proposes a solution to this problem, based on a more biological noise model.]
MATLAB code for estimating ICA, ISA, and TICA bases from image data is also available (by P. O. Hoyer).
A MATLAB package for the FastICA algorithm is also available.
Project home pageA. Hyvärinen.
Fast and Robust Fixed-Point Algorithms for Independent Component Analysis. IEEE Transactions on Neural Networks 10(3):626-634, 1999.
Abstract
Postscript
gzipped PostScript
html pdf
[The fundamental paper on the FastICA algorithm, which is a computationally very efficient method for performing ICA. This article is based on the TechRep "ICA by Minimization of Mutual Information".]
A. Hyvärinen and E. Oja.
Independent Component Analysis: Algorithms and Applications.
Neural Networks, 13(4-5):411-430, 2000.
html (and Japanese version)
Postscript gzipped PostScript
pdf
[A tutorial text on ICA in general, and FastICA in particular.]
A. Hyvärinen and E. Oja. A Fast Fixed-Point Algorithm for Independent
Component Analysis.
Neural Computation, 9(7):1483-1492, 1997.
Abstract
Postscript
gzipped PostScript.
[Introduced the original, cumulant-based, form of the FastICA
algorithm.]
A. Hyvärinen. New Approximations of Differential Entropy
for Independent Component Analysis and Projection Pursuit. In
Advances in Neural Information Processing Systems 10 (NIPS*97), pp. 273-279, MIT Press, 1998.
Abstract
Postscript
gzipped PostScript
pdf.
Related TechRep: Postscript
gzipped PostScript.
[Introduced approximations of differential entropy used in the derivation of the FastICA algorithm in the IEEE Transactions paper above.]
E. Bingham and A. Hyvärinen
A fast fixed-point algorithm for independent component analysis of complex-valued signals.
Int. J. of Neural Systems, 10(1):1-8, 2000.
Postscript
gzipped
PostScript
Matlab code
[A version of FastICA for complex-valued data.]
A. Hyvärinen. One-Unit Contrast Functions for Independent
Component Analysis: A Statistical Analysis. In Neural Networks
for Signal Processing VII (Proc. IEEE NNSP Workshop '97, Amelia Island,
Florida), pp. 388--397, 1997.
Abstract
Postscript
gzipped PostScript.
[Gives a statistical analysis of the maximum nongaussianity framework used in FastICA and some other ICA algorithms given below.]
A. Hyvärinen. The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis. Neural Processing Letters, 10(1):1-5.
Abstract
Postscript
gzipped PostScript.
[Shows how the FastICA algorithms can be interpreted as maximum likelihood estimation.]
A. Hyvärinen. Sparse Code Shrinkage: Denoising of Nongaussian
Data by Maximum Likelihood Estimation.
Neural Computation, 11(7):1739--1768, 1999.
Abstract
Postscript gzipped
PostScript
[Describes sparse code shrinkage, which is a new method for denoising of images etc. It is a kind of a combination of independent component analysis and wavelet shrinkage ideas.]
A. Hyvärinen, P. Hoyer and E. Oja. Image Denoising by Sparse Code Shrinkage. In S. Haykin and B. Kosko (eds), Intelligent Signal Processing, IEEE Press, 2001.
Postscript gzipped
PostScript
[Describes sparse code shrinkage in much more detail.]
A. Hyvärinen. Independent Component Analysis in the Presence
of Gaussian Noise by Maximizing Joint Likelihood. Neurocomputing,
22:49-67, 1998.
Abstract
Postscript
gzipped PostScript.
[Older work on noisy independent component analysis and its connections
to competitive learning.]
A. Hyvärinen. Gaussian Moments for Noisy Independent Component Analysis.
IEEE Signal Processing Letters, 6(6):145--147, 1999.
Abstract
Postscript
gzipped PostScript.
Longer paper with proofs (Proc. ISCAS'99): gzipped PostScript.
[Shows how to modify the FastICA algorithm obtain consistent estimators when the data is corrupted by Gaussian noise. Introduces the concept of Gaussian moments.]
A. Hyvärinen and M. Inki.
Estimating overcomplete independent component bases for image windows.
Journal of Mathematical Imaging and Vision, 17:139-152, 2002.
Abstract
Postscript gzipped
PostScript pdf
[Discusses different methods for overcomplete basis estimation from images,
and proposes two new, computationally efficient algorithms.]
A. Hyvärinen, R. Cristescu and E. Oja.
A fast algorithm for estimating overcomplete ICA bases for
image windows.
In Proc. Int. Joint Conf on Neural Networks,
Washington D.C., 1999.
Abstract
Postscript gzipped
PostScript
[A version of FastICA that estimates overcomplete bases, especially suitable for image data.]
A. Hyvärinen.
Complexity Pursuit: Separating interesting components from time-series.
Neural Computation, 13(4):883--898, 2001.
Abstract
Postscript gzipped PostScript
[Introduces the concept of complexity pursuit, which means finding projections of time series (signals) that have minimum complexity. Also introduces simple approximations of complexity that take into account both nongaussianity and autocorrelations.]
A. Hyvärinen.
Blind source separation by nonstationarity of variance: A cumulant-based approach.
IEEE Trans. on Neural Networks, 12(6):1471-1474, 2001.
Abstract
Postscript gzipped PostScript
pdf
[Formulates the less-known separation criterion on variance nonstationarity using cumulants, and proposes a fast fixed-point algorithm.]
A. Hyvärinen. Independent Component Analysis
for Time-dependent Stochastic Processes. In Proc. Int. Conf. on Artificial Neural Networks (ICANN'98),
Skövde, Sweden, pp. 541-546, 1998.
Abstract
Postscript
gzipped PostScript
[This paper shows that in ICA, it is often useful to preprocess the data by computing the innovation processes.]
A. Hyvärinen and E. Oja. Independent Component Analysis by General
Non-linear Hebbian-like Learning Rules. Signal Processing,
64(3):301-313, 1998.
Abstract
Postscript
gzipped PostScript .
[Introduces one-unit adaptive algorithms related to FastICA. Shows how to estimate a coefficient that allows the estimation of both sub- and super-Gaussian independent component using a single nonlinearity.]
A. Hyvärinen. One-Unit Learning Rules for Independent
Component Analysis.
In Advances in Neural Information Processing Systems 9 (NIPS*96),
MIT Press, pp. 480--486, 1997.
Abstract
Postscript
gzipped PostScript.
[A nice if somewhat out-dated overview on the one-unit ICA algorithms,
including fixed-point and adaptive versions.]
A. Hyvärinen and E. Oja. Simple Neuron Models for Independent
Component Analysis. Int. Journal of Neural Systems, 7(6):671-687, 1996.
Postscript
gzipped PostScript.
[Some older work, rather out-dated.]
A. Hyvärinen and P. Pajunen. Nonlinear Independent Component Analysis:
Existence and Uniqueness results. Neural Networks 12(3): 429--439, 1999.
Abstract
Postscript
gzipped PostScript
[Shows that the solution of the nonlinear ICA problem is highly non-unique, and proposes a restriction of the model that does make the solution unique.]
P. Pajunen, A. Hyvärinen and J. Karhunen. Non-Linear Blind Source
Separation by Self-Organizing Maps. In Proc. Int. Conf. on Neural
Information Processing, Hong Kong, pp. 1207-1210, 1996.
Postscript
gzipped PostScript.
[Describes a simple if limited method for doing nonlinear (nonparametric) ICA for subgaussian signals.]
A. Hyvärinen and R. Karthikesh.
Imposing sparsity on the mixing matrix in independent component analysis.
Neurocomputing, 49:151-162, 2002 (Special Issue on ICA and BSS).
Abstract
Postscript gzipped PostScript
[Shows how to implement the prior information on the sparsity of the mixing matrix in a very simple way as conjugate priors.]
A. Hyvärinen, J. Särelä and R. Vigário.
Bumps and Spikes: Artifacts Generated by Independent Component Analysis with Insufficient Sample Size. In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA'99), pp. 425-429, Aussois, France, 1999.
Abstract
Postscript (a very large file) gzipped PostScript.
[Describes the phenomenon of overlearning in ICA.]
A. Hyvärinen and E. Bingham.
Connection between multi-layer perceptrons and regression using independent component analysis. Neurocomputing, 50(C):211-222, 2003.
Abstract
Postscript gzipped
PostScript
[Shows that MLP's can be interpreted as estimating an ICA model for the data, and doing regression using that model.]
J. Himberg and A. Hyvärinen.
Independent component analysis for binary data: An experimental study
.
In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA2001), San Diego, California, 2001.
Postscript gzipped
PostScript
[Shows how to do ICA on binary data using ordinary FastICA.]
A. Hyvärinen and E. Oja.
Independent Component Analysis: Algorithms and Applications.
Neural Networks, 13(4-5):411-430, 2000.
html (and Japanese version)
Postscript gzipped PostScript
pdf
[A tutorial text on ICA in general, and FastICA in particular (already mentioned above.)]
A. Hyvärinen. Survey on Independent Component Analysis.
Neural Computing Surveys 2:94--128, 1999.
html 
Postscript
gzipped PostScript
pdf
[A comprehensive review on ICA estimation methods.]
A. Hyvärinen and Y. Kano.
Independent component analysis for non-normal factor analysis.
Proc. International Meeting of the Psychometric Society (IMPS2001), Osaka, Japan.
Postscript pdf
[Tutorial of ICA for people who already are familiar with factor analysis.]