Näitä sivuja ei päivitetä enää. Ole hyvä
ja katso tietojenkäsittelytieteen laitoksen WWW-sivuja:
http://ics.tkk.fi/fi/studies/.
These pages are not any more updated.
Please, see web pages of Department of Information and Computer Science (ICS):
http://ics.tkk.fi/en/studies/.
T-61.5130 Machine Learning and Neural Networks (5 cr)
Autumn 2007, Lecture Period II.
Lecturer: Prof. Juha Karhunen. Course assistant: M.Sc. Matti Pöllä.
Background
This new course was lectured for the first time on Teaching period II in
autumn 2007. The course and its materials are completely in English as a part of the
new English
Macadamia study programme.
The participants should have a knowledge of
basic university mathematics, including fundamentals of calculus, linear algebra,
and probability theory. Our laboratory's first machine learning
course
T-61.3050 Machine Learning:
Basic Principles provides a useful but not necessary background to this second
machine learning course, which is fairly independent also from our laboratory's
third machine learning course
T-61.5140
Machine Learning: Advanced Probabilistic Methods.
Formally, this course replaces our old course T-61.5030
Advanced Course in Neural Computing. In practice, its contents cover the central parts
of our earlier courses T-61.3030 Principles of Neural Computing and T-61.5030 Advanced
Course in Neural Computing. A more detailed description of the relationships of our
courses can be found on
this web page.
This course can be included into graduate studies, too.
Contents
The following matters are discussed in this course:
- Introduction to neural networks, examples of their applications.
- Models of neuron, activation functions, network architectures.
- Single neuron models and learning rules: least-mean squares (LMS) algorithm,
basic perceptron, their weaknesses.
- Hebbian learning and principal component analysis (PCA), preprocessing of data.
- Feedforward multilayer perceptron (MLP) networks, backpropagation learning algorithms,
their properties and some improvements.
- Advanced optimization algorithms for multilayer perceptron networks: conjugate gradient
algorithm, Levenberg-Marquardt algorithm.
- Model assessment and selection: generalization, overlearning, regularization,
bias-variance decomposition, validation and cross-validation.
- Radial-basis function (RBF) neural networks and their learning algorithms.
- Support vector machines for classification and nonlinear regression.
- Independent component analysis (ICA): basic principles, criteria, learning algorithms,
and some applications.
- Self-organizing maps (SOM) and learning vector quantization (LVQ).
- Processing of temporal information in feedforward networks,
simple recurrent network.
Material
The course is based loosely on the book F. Ham and I. Kostanic, "Principles of
Neurocomputing for Science & Engineering", McGraw-Hill 2001, Chapters 2-5. These
chapters deal with fundamentals of neural computing, as well as various neural
network architectures and their learning rules. This book is complemented with
the lecturer's own slides especially on on model assessment and selection.
Support vector machines are covered from Chapter 6 of the textbook S. Haykin,
"Neural Networks: A Comprehensive Foundation", 2nd ed., Prentice-Hall 1998.
Independent component analysis is discussed based on the tutorial article
A. Hyvärinen and E. Oja, "Independent component analysis: algorithms and applications",
Neural Networks, 2000, pp. 411-430.
The problem sheets and solutions of the exercises are available
here. The exercises, their solutions, examination
requirements, and lecture slides written in English, as well as the additional lecture
material used is copied to the participants as lecture notes via Edita Prima Oy.
Lectures and exercises
The lectures are on Mondays 12-14 o'clock in the lecture hall TU1
(in the TUAS Department building), and Wednesdays 10-12 o'clock in the lecture hall
T2 (in the Dept. of Computer Science and Eng. building).
Solutions of the
exercises are presented on Thursdays
12-14 o'clock in the lecture hall AS2 (in the TUAS building),
and on Tuesdays 14-16 o'clock in the lecture hall T3. The lectures and exercises cover
the teaching period II (from 1st November to 13th December) in autumn 2007.
The exercises consist of standard exercise problems, and computer problems (for
example, running given algorithms in example problems). Their solutions are demonstrated
by the course assistant in the exercises. Furthermore, selected demos are presented
in context with the exercises at suitable places.
The lectures are given by Prof.
Juha Karhunen. He can be met
during the lectures, or by email: Juha.Karhunen@tkk.fi (Tel. 09-451 3270, mobile
0400-817 276, Room B327 in Computer Science and Engineering Dept. House).
The
exercises are given by the course assistant, M.Sc.
Matti Pöllä. He can be met
during the exercises, or by email: matti.polla@tkk.fi (Tel. 09-451 5115, room B332
in Computer Science and Engineering Dept. House).
Computer assignment
Furthermore, the course includes one
computer assignment for each
participant. To pass the course, the participants must pass the examination and
perform acceptably the computer assignment allocated to him or her. The
deadline for the course assignment report is January 31st 2008.
Examinations
The first examination was arranged on Wednesday 19th December 13-16 o'clock,
in the lecture hall T1. The second exam was held on Friday 11th January 2008 in
the lecture hall T1, and the third one will be held on Saturday 8th March 2008.
Notice: Changes are still possible! Please enroll to the examination
via wwwtopi a week before the examination at the latest if possible.
Examination requirements.
Results of the examination on 19th December 2007 are now on the
notice board of the course. They are briefly summarized
here.
Results of the examination on 11th January 2008 are now on the
notice board of the course. They are briefly summarized
here.
Notices and enrollment
Announcements concerning this course are given on the lectures and exercises,
and on the
web page of the
course. The notice board of the course is in the 3rd floor of the T-building.
Results of examinations will appear there. Please enroll to the course using wwwtopi,
or if this is not possible by e-mail to the lecturer.
Lecture slides
Each lecture below contains slides on one major topic. The number of slides per
topic varies a lot, and their presentation may in practice take more or less than
one 2 x 45 minutes long standard oral lecture. The lecture slides are available in
.pdf form both as full color versions for viewing, and black-and-white (B/W) versions for
printing.
General information about the course
Lecture 1, B/W version
Lecture 2, B/W version
Lecture 3, B/W version
Lecture 4, B/W version
Lecture 5, B/W version
Lecture 6, B/W version
Lecture 7, B/W version
Lecture 8, B/W version
Lecture 9, B/W version
Lecture 10, B/W version
Lecture 11, B/W version
Lecture 12, B/W version
Otaniemi, January 29, 2008
Prof. Juha Karhunen