Näitä sivuja ei päivitetä enää. Ole hyvä ja katso tietojenkäsittelytieteen laitoksen WWW-sivuja: http://ics.tkk.fi/fi/studies/.

These pages are not any more updated. Please, see web pages of Department of Information and Computer Science (ICS): http://ics.tkk.fi/en/studies/.

T-61.5130 Machine Learning and Neural Networks (5 cr)

Autumn 2007, Lecture Period II.

Lecturer: Prof. Juha Karhunen. Course assistant: M.Sc. Matti Pöllä.

Background

This new course was lectured for the first time on Teaching period II in autumn 2007. The course and its materials are completely in English as a part of the new English Macadamia study programme. The participants should have a knowledge of basic university mathematics, including fundamentals of calculus, linear algebra, and probability theory. Our laboratory's first machine learning course T-61.3050 Machine Learning: Basic Principles provides a useful but not necessary background to this second machine learning course, which is fairly independent also from our laboratory's third machine learning course T-61.5140 Machine Learning: Advanced Probabilistic Methods. Formally, this course replaces our old course T-61.5030 Advanced Course in Neural Computing. In practice, its contents cover the central parts of our earlier courses T-61.3030 Principles of Neural Computing and T-61.5030 Advanced Course in Neural Computing. A more detailed description of the relationships of our courses can be found on this web page. This course can be included into graduate studies, too.

The following matters are discussed in this course:

- Introduction to neural networks, examples of their applications.
- Models of neuron, activation functions, network architectures.
- Single neuron models and learning rules: least-mean squares (LMS) algorithm, basic perceptron, their weaknesses.
- Hebbian learning and principal component analysis (PCA), preprocessing of data.
- Feedforward multilayer perceptron (MLP) networks, backpropagation learning algorithms, their properties and some improvements.
- Advanced optimization algorithms for multilayer perceptron networks: conjugate gradient algorithm, Levenberg-Marquardt algorithm.
- Model assessment and selection: generalization, overlearning, regularization, bias-variance decomposition, validation and cross-validation.
- Radial-basis function (RBF) neural networks and their learning algorithms.
- Support vector machines for classification and nonlinear regression.
- Independent component analysis (ICA): basic principles, criteria, learning algorithms, and some applications.
- Self-organizing maps (SOM) and learning vector quantization (LVQ).
- Processing of temporal information in feedforward networks, simple recurrent network.

Material

The course is based loosely on the book F. Ham and I. Kostanic, "Principles of Neurocomputing for Science & Engineering", McGraw-Hill 2001, Chapters 2-5. These chapters deal with fundamentals of neural computing, as well as various neural network architectures and their learning rules. This book is complemented with the lecturer's own slides especially on on model assessment and selection. Support vector machines are covered from Chapter 6 of the textbook S. Haykin, "Neural Networks: A Comprehensive Foundation", 2nd ed., Prentice-Hall 1998. Independent component analysis is discussed based on the tutorial article A. Hyvärinen and E. Oja, "Independent component analysis: algorithms and applications", Neural Networks, 2000, pp. 411-430.

The problem sheets and solutions of the exercises are available here. The exercises, their solutions, examination requirements, and lecture slides written in English, as well as the additional lecture material used is copied to the participants as lecture notes via Edita Prima Oy.

Lectures and exercises

The lectures are on Mondays 12-14 o'clock in the lecture hall TU1 (in the TUAS Department building), and Wednesdays 10-12 o'clock in the lecture hall T2 (in the Dept. of Computer Science and Eng. building). Solutions of the exercises are presented on Thursdays 12-14 o'clock in the lecture hall AS2 (in the TUAS building), and on Tuesdays 14-16 o'clock in the lecture hall T3. The lectures and exercises cover the teaching period II (from 1st November to 13th December) in autumn 2007.

The exercises consist of standard exercise problems, and computer problems (for example, running given algorithms in example problems). Their solutions are demonstrated by the course assistant in the exercises. Furthermore, selected demos are presented in context with the exercises at suitable places.

The lectures are given by Prof. Juha Karhunen. He can be met during the lectures, or by email: Juha.Karhunen@tkk.fi (Tel. 09-451 3270, mobile 0400-817 276, Room B327 in Computer Science and Engineering Dept. House).

The exercises are given by the course assistant, M.Sc. Matti Pöllä. He can be met during the exercises, or by email: matti.polla@tkk.fi (Tel. 09-451 5115, room B332 in Computer Science and Engineering Dept. House).

Computer assignment

Furthermore, the course includes one computer assignment for each participant. To pass the course, the participants must pass the examination and perform acceptably the computer assignment allocated to him or her. The deadline for the course assignment report is January 31st 2008.

Examinations

The first examination was arranged on Wednesday 19th December 13-16 o'clock, in the lecture hall T1. The second exam was held on Friday 11th January 2008 in the lecture hall T1, and the third one will be held on Saturday 8th March 2008. Notice: Changes are still possible! Please enroll to the examination via wwwtopi a week before the examination at the latest if possible.

Examination requirements.

Results of the examination on 19th December 2007 are now on the notice board of the course. They are briefly summarized here.
Results of the examination on 11th January 2008 are now on the notice board of the course. They are briefly summarized here.

Notices and enrollment

Announcements concerning this course are given on the lectures and exercises, and on the web page of the course. The notice board of the course is in the 3rd floor of the T-building. Results of examinations will appear there. Please enroll to the course using wwwtopi, or if this is not possible by e-mail to the lecturer.

Lecture slides

Each lecture below contains slides on one major topic. The number of slides per topic varies a lot, and their presentation may in practice take more or less than one 2 x 45 minutes long standard oral lecture. The lecture slides are available in .pdf form both as full color versions for viewing, and black-and-white (B/W) versions for printing.

General information about the course
Lecture 1, B/W version
Lecture 2, B/W version
Lecture 3, B/W version
Lecture 4, B/W version
Lecture 5, B/W version
Lecture 6, B/W version
Lecture 7, B/W version
Lecture 8, B/W version
Lecture 9, B/W version
Lecture 10, B/W version
Lecture 11, B/W version
Lecture 12, B/W version

Otaniemi, January 29, 2008

Prof. Juha Karhunen