T. Kohonen: Self-Organizing Maps

Contents

  • 1. Mathematical Preliminaries 1
    • 1.1 Mathematical Concepts and Notations 2
      • Vector Space Concepts 2
      • Matrix Notations 8
      • Eigenvectors and Eigenvalues of Matrices 11
      • Further Properties of Matrices 13
      • On Matrix Differential Calculus 15

    • 1.2 Distance Measures for Patterns 17
      • Measures of Similarity and Distance in Vector Spaces 17
      • Measures of Similarity and Distance Between Symbol Strings 21
      • Averages over Nonvectorial Variables 28

    • 1.3 Statistical Pattern Analysis 29
      • Basic Probabilistic Concepts 29
      • Projection Methods 34
      • Supervised Classification 39
      • Unsupervised Classification 44

    • 1.4 The Subspace Methods of Classification 46
      • The Basic Subspace Method 46
      • Adaptation of a Model Subspace to Input Subspace 49
      • The Learning Subspace Method (LSM) 53

    • 1.5 Vector Quantization 59
      • Definitions 59
      • Derivation of the VQ Algorithm 60
      • Point Density in VQ 62

    • 1.6 Dynamically Expanding Context 64
      • Setting Up the Problem 66
      • Automatic Determination of Context-Independent Productions 66
      • Conflict Bit 67
      • Construction of Memory for the Context-Dependent Productions 68
      • The Algorithm for the Correction of New Strings 68
      • Estimation Procedure for Unsuccessful Searches 69
      • Practical Experiments 69

  • 2. Neural Modeling 71
    • 2.1 Models, Paradigms, and Methods 71
    • 2.2 A History of Some Main Ideas in Neural Modeling 72
    • 2.3 Issues on Artificial Intelligence 75
    • 2.4 On the Complexity of Biological Nervous Systems 76
    • 2.5 What the Brain Circuits Are Not 78
    • 2.6 Relation Between Biological and Artificial Neural Networks 79
    • 2.7 What Functions of the Brain Are Usually Modeled? 81
    • 2.8 When Do We Have to Use Neural Computing? 81
    • 2.9 Transformation, Relaxation, and Decoder 82
    • 2.10 Categories of ANNs 85
    • 2.11 A Simple Nonlinear Dynamic Model of the Neuron 87
    • 2.12 Three Phases of Development of Neural Models 89
    • 2.13 Learning Laws 91
      • Hebb's Law 91
      • The Riccati-Type Learning Law 92
      • The PCA-Type Learning Law 95

    • 2.14 Some Really Hard Problems 96

    • 2.15 Brain Maps 99

  • 3. The Basic SOM 105
    • 3.1 A Qualitative Introduction to the SOM 106
    • 3.2 The Original Incremental SOM Algorithm 109
    • 3.3 The ``Dot-Product SOM" 115
    • 3.4 Other Preliminary Demonstrations of Topology-Preserving Mappings 116
      • Ordering of Reference Vectors in the Input Space 116
      • Demonstrations of Ordering of Responses in the Output Space 120

    • 3.5 Basic Mathematical Approaches to Self-Organization 127
      • One-Dimensional Case 128
      • Constructive Proof of Ordering of Another One-dimensional SOM 132

    • 3.6 The Batch Map 138

    • 3.7 Initialization of the SOM Algorithms 142

    • 3.8 On the ``Optimal" Learning-Rate Factor 143

    • 3.9 Effect of the Form of the Neighborhood Function 145

    • 3.10 Does the SOM Algorithm Ensue from a Distortion Measure? 146

    • 3.11 An Attempt to Optimize the SOM 148

    • 3.12 Point Density of the Model Vectors 152
      • Earlier Studies 152
      • Numerical Check of Point Densities in a Finite One-Dimensional SOM 153

    • 3.13 Practical Advice for the Construction of Good Maps 159

    • 3.14 Examples of Data Analyses Implemented by the SOM 161
      • Attribute Maps with Full Data Matrix 161
      • Case Example of Attribute Maps Based on Incomplete Data Matrices (Missing Data): "Poverty Map" 165

    • 3.15 Using Gray Levels to Indicate Clusters in the SOM 165

    • 3.16 Interpretation of the SOM Mapping 166
      • ``Local Principal Components'' 166
      • Contribution of a Variable to Cluster Structures 169

    • 3.17 Speedup of SOM Computation 170
      • Shortcut Winner Search 170
      • Increasing the Number of Units in the SOM 172
      • Smoothing 175
      • Combination of Smoothing, Lattice Growing, and SOM Algorithm 176

  • 4.Physiological Interpretation of SOM 177
    • 4.1 Conditions for Abstract Feature Maps in the Brain 177
    • 4.2 Two Different Lateral Control Mechanisms 178
      • The WTA Function, Based on Lateral Activity Control 179
      • Lateral Control of Plasticity 184

    • 4.3 Learning Equation 185

    • 4.4 System Models of SOM and Their Simulations 185

    • 4.5 Recapitulation of the Features of the Physiological SOM Model 188

    • 4.6 Similarities Between the Brain Maps and Simulated Feature Maps 188
      • Magnification 189
      • Imperfect Maps 189
      • Overlapping Maps 189

  • 5. Variants of SOM 191
    • 5.1 Overview of Ideas to Modify the Basic SOM 191
    • 5.2 Adaptive Tensorial Weights 194
    • 5.3 Tree-Structured SOM in Searching 197
    • 5.4 Different Definitions of the Neighborhood 198
    • 5.5 Neighborhoods in the Signal Space 200
    • 5.6 Dynamical Elements Added to the SOM 204
    • 5.7 The SOM for Symbol Strings 205
      • Initialization of the SOM for Strings 205
      • The Batch Map for Strings 206
      • Tie-Break Rules 206
      • A Simple Example: The SOM of Phonemic Transcriptions 207

    • 5.8 Operator Maps 207

    • 5.9 Evolutionary-Learning SOM 211
      • Evolutionary-Learning Filters 211
      • Self-Organization According to a Fitness Function 212

    • 5.10 Supervised SOM 215

    • 5.11 The Adaptive-Subspace SOM (ASSOM) 216
      • The Problem of Invariant Features 216
      • Relation Between Invariant Features and Linear Subspaces 218
      • The ASSOM Algorithm 222
      • Derivation of the ASSOM Algorithm by Stochastic Approximation 226
      • ASSOM Experiments 228

    • 5.12 Feedback-Controlled Adaptive-Subspace SOM (FASSOM) 242

  • 6. Learning Vector Quantization 245
    • 6.1 Optimal Decision 245
    • 6.2 The LVQ1 246
    • 6.3 The Optimized-Learning-Rate LVQ1 (OLVQ1) 250
    • 6.4 The Batch-LVQ1 251
    • 6.5 The Batch-LVQ1 for Symbol Strings 252
    • 6.6 The LVQ2 (LVQ2.1) 252
    • 6.7 The LVQ3 253
    • 6.8 Differences Between LVQ1, LVQ2 and LVQ3 254
    • 6.9 General Considerations 254
    • 6.10 The Hypermap-Type LVQ 256
    • 6.11 The ``LVQ-SOM'' 261

  • 7. Applications 263
    • 7.1 Preprocessing of Optic Patterns 264
      • Blurring 265
      • Expansion in Terms of Global Features 266
      • Spectral Analysis 266
      • Expansion in Terms of Local Features (Wavelets) 267
      • Recapitulation of Features of Optic Patterns 267

    • 7.2 Acoustic Preprocessing 268

    • 7.3 Process and Machine Monitoring 269
      • Selection of Input Variables and Their Scaling 269
      • Analysis of Large Systems 270

    • 7.4 Diagnosis of Speech Voicing 274

    • 7.5 Transcription of Continuous Speech 274

    • 7.6 Texture Analysis 280

    • 7.7 Contextual Maps 281
      • Artifically Generated Clauses 283
      • Natural Text 285

    • 7.8 Organization of Large Document Files 286
      • Statistical Models of Documents 286
      • Construction of Very Large WEBSOM Maps by the Projection Method 292
      • The WEBSOM of All Electronic Patent Abstracts 296

    • 7.9 Robot-Arm Control 299
      • Simultaneous Learning of Input and Output Parameters 299
      • Another Simple Robot-Arm Control 303

    • 7.10 Telecommunications 304
      • Adaptive Detector for Quantized Signals 304
      • Channel Equalization in the Adaptive QAM 305
      • Error-Tolerant Transmission of Images by a Pair of SOMs 306

    • 7.11 The SOM as an Estimator 308
      • Symmetric (Autoassociative) Mapping 308
      • Asymmetric (Heteroassociative) Mapping 309

  • 8. Software Tools for SOM 311
    • 8.1 Necessary Requirements 311
    • 8.2 Desirable Auxiliary Features 313
    • 8.3 SOM Program Packages 315
      • SOM_PAK 315
      • SOM Toolbox 317
      • Nenet (Neural Networks Tool) 318
      • Viscovery SOMine 318

    • 8.4 Examples of the Use of SOM_PAK 319
      • File Formats 319
      • Description of the Programs in SOM_PAK 322
      • A Typical Training Sequence 326

    • 8.5 Neural-Networks Software with the SOM Option 327

  • 9.Hardware for SOM 329
    • 9.1 An Analog Classifier Circuit 329
    • 9.2 Fast Digital Classifier Circuits 332
    • 9.3 SIMD Implementation of SOM 337
    • 9.4 Transputer Implementation of SOM 339
    • 9.5 Systolic-Array Implementation of SOM 341
    • 9.6 The COKOS Chip 342
    • 9.7 The TInMANN Chip 342
    • 9.8 NBISOM_25 Chip 344

  • 10.An Overview of SOM Literature 347
    • 10.1 Books and Review Articles 347
    • 10.2 Early Works on Competitive Learning 348
    • 10.3 Status of the Mathematical Analyses 349
      • Zero-Order Topology (Classical VQ) Results. 349
      • Alternative Topological Mappings 350
      • Alternative Architectures 350
      • Functional Variants 351
      • Theory of the Basic SOM 352

    • 10.4 The Learning Vector Quantization 358

    • 10.5 Diverse Applications of SOM 358
      • Machine Vision and Image Analysis 358
      • Optical Character and Script Reading 360
      • Speech Analysis and Recognition 360
      • Acoustic and Musical Studies 361
      • Signal Processing and Radar Measurements 362
      • Telecommunications 362
      • Industrial and Other Real-World Measurements 362
      • Process Control 363
      • Robotics 364
      • Electronic-Circuit Design 364
      • Physics 364
      • Chemistry 365
      • Biomedical Applications Without Image Processing 365
      • Neurophysiological Research 366
      • Data Processing and Analysis 366
      • Linguistic and AI Problems 367
      • Mathematical and Other Theoretical Problems 368

    • 10.6 Applications of LVQ 369
    • 10.7 Survey of SOM and LVQ Implementations 370

  • 11.Glossary of "Neural" Terms 373


  • References 403


  • Index 487

Go back to the book


http://www.cis.hut.fi/research/som-research/book/toc.shtml
webmaster@mail.cis.hut.fi
Monday, 29-Jan-2001 16:54:47 EET