Laboratory of Computer and Information Science / Neural Networks Research Centre CIS Lab Helsinki University of Technology

Content-Based Image and Information Retrieval

The scientifically challenging objective of content-based image retrieval (CBIR) is the autonomous adaptation to the user's preferences and conceptions regarding the relevance and similarity of images. This can be achieved by examining the user's reactions on the retrieval results in the form of relevance feedback.

Main research areas

We have recently developed a framework for content-based information retrieval research named the PicSOM system. The research has been and will be three-fold: algorithm development, international evaluations and applications. We will continue to study the basic principles of effective algorithms that implement the Query by Example paradigm in connection with relevance feedback from the user.

We will also study other paradigms such as query by sketch and query by pointing. These modalities will be required in our joint multimodal interfaces initiative. A key concept in the methodological developments will be the utilization of automatic segmentation or location of interest points in the images or other information objects. After the segmentation, relevance feedback from the user and other forms of auxiliary or cross-modal information can be focused on specific subparts of the objects.

Various real-life image retrieval tasks will be studied further. We have already attained very good results in a joint project with ABB, in which the PicSOM system has been applied to recognition of defects in running paper and metal webs.

Snapshot of PicSOM interface

Future plans

We will continue to broaden the studied domain to include diverse multimodal forms of information, e.g. sound, video, and various multi-part data types. For such experiments we have already incorporated the standardized MPEG-7 visual descriptors in the PicSOM system. Of special interest to us is the interplay between different modalities and how annotations and relevance assessments can be extrapolated over the modality boundary. Specific topics on which we will focus in the years 2006-2008 include the following:

Recent publications












Erkki Oja Jorma Laaksonen Markus Koskela Ville Viitaniemi Zhirong Yang Mats Sjöberg He Zhang

You are at: CISResearchMultimodal Interfaces → Content-Based Information Retrieval

Page maintained by picsom at, last updated Friday, 03-Dec-2010 16:18:27 EET