PROACTIVE INFORMATION RETRIEVAL

This research area was started within PRIMA project in 2003 and continues within later research projects.

Motivation

Successful proactivity, i.e. anticipation, in varying contexts requires generalization from past experience. Generalization, on its part, requires suitable powerful (stochastic) models and a collection of data about relevant past history to learn the models.

Objectives

The goal of PRIMA project is to build statistical machine learning models that learn from the actions of people to model their intentions and actions. The models are used for disambiguating the users' vague commands and anticipating their actions. The actions and interests are monitored by measuring eye fixations and movements that exhibit both voluntary and involuntary signs of the users' intentions. (An overview)

On-Going Projects

• PinView is a EU FP7 funded collaborative project that aims in building a personal information navigator for retrieving multimedia from unannotated databases.
  
• Urban contextual information interfaces with multimodal augmented reality (UI-ART) is funded by the Multidisciplinary Institute of Digitalisation and Energy (MIDE) programme.
  

Earlier Projects

• Methods for Fusing Eye Movements and Text Content for Information Retrieval
     in the Pump Priming Programme,
     funded by The PASCAL network of excellence
     (Pattern Analysis, Statistical Modellling and Computational Learning).
• Proactive Information Retrieval by Adaptive Models of User's Attention and Interests (PRIMA)
     in the research programme on proactive information technology Proaktiivinen tietotekniikka (PROACT)
     funded by The Academy of Finland.

---

Researchers

Adaptive Informatics Research Centre
Helsinki University of Technology, Finland
Research group on Statistical Machine Learning and Bioinformatics

•    Professor Samuel Kaski
  
•    Ph.D. Mehmet Gönen
  
•    D.Sc. Arto Klami
  
•    M.Sc. Antti Ajanki
  
•    M.Sc. Melih Kandemir
  
•    M.Sc. Maija Nevala
  
•    Chiwei Wang
  

Affiliated researchers:

•    Ph.D. Kai Puolamäki
  
  

---

Challenge

INFERRING RELEVANCE FROM EYE MOVEMENTS CHALLENGE

1 March - 30 September 2005
The objective of the Challenge is to predict from eye movement data whether a reader finds a text relevant.

NIPS 2005 Workshop

WORKSHOP ON MACHINE LEARNING FOR IMPLICIT FEEDBACK AND USER MODELING

December 10 at Whistler, Canada
The workshop is intended to gather together machine learning researchers interested in a new promising and challenging research area. Half of the papers will introduce best-performing solutions of the Challenge, the rest will broaden the scope towards other implicit feedback signals, and towards more general problems of user modeling.

---

Media & Events

• Voiko kone tulevaisuudessa arvata tahtosi?
  Apropos 1/2005, Magazine of the Academy of Finland (in Finnish)
  
  
• The computer serves its user better by tracking eye gaze
  Press release April 13th, 2005
  
  
• Selected images and videos
  
  

---

Publications

2010

 

1. Arto Klami. Inferring Task-relevant Image Regions from Gaze Data. To appear in Machine Learning for Signal Processing 2010.


2. Antti Ajanki, Mark Billinghurst, Melih Kandemir, Samuel Kaski, Markus Koskela, Mikko Kurimo, Jorma Laaksonen, Kai Puolamäki and Timo Tossavainen: Ubiquitous Contextual Information Access with Proactive Retrieval and Augmentation. The fourth International Workshop on Ubiquitous Virtual Reality (IWUVR), 2010. (abstract, pdf)
   
   
3. Antti Ajanki, Mark Billinghurst, Hannes Gamper, Toni Järvenpää, Melih Kandemir, Samuel Kaski, Markus Koskela, Mikko Kurimo, Jorma Laaksonen, Kai Puolamäki, Teemu Ruokolainen and Timo Tossavainen: Contextual Information Access with Augmented Reality. In Proceedings of IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 2010. (abstract, pdf)
   
   
4. Peter Auer, Zakria Hussain, Samuel Kaski, Arto Klami, Jussi Kujala, Jorma Laaksonen, Alex P. Leung, Kitsuchart Pasupa, John Shawe-Taylor: Pinview: Implicit Feedback in Content-Based Image Retrieval. In Proceedings of Workshop on Applications of Pattern Analysis , published by JMLR Workshop and Conference Proceedings, 2010. (abstract, pdf)

2009

 

1. Antti Ajanki, David R. Hardoon, Samuel Kaski, Kai Puolamäki and John Shawe-Taylor: Can Eyes Reveal Interest? - Implicit Queries from Gaze Patterns. User Modeling and User-Adapted Interaction, 2009. (abstract, DOI)
   
   
2. László Kozma, Arto Klami, and Samuel Kaski: GaZIR: Gaze-based Zooming Interface for Image Retrieval. In Proceedings of 11th Conference on Multimodal Interfaces and The Sixth Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI), 2009. (abstract, pdf)
   
   
3. Kitsuchart Pasupa, Craig Saunders, Sandor Szedmak, Arto Klami, Samuel Kaski and Steve Gunn. Learning to Rank Images from Eye Movements. In Proceedings of ICCV'2009 Workshop on Human-Computer Interaction (HCI'2009), pp. 2009--2016, 2009. (pdf)

2008

 

1. Eerika Savia, Kai Puolamäki, and Samuel Kaski: Latent Grouping Models for User Preference Prediction. Published online in Machine Learning, September 2008. In Press. (abstract, DOI)
   
   
2. Kai Puolamäki, Antti Ajanki, and Samuel Kaski: Learning to Learn Implicit Queries from Gaze Patterns. International Conference on Machine Learning (ICML 2008), Helsinki, Finland, July 5-9, 2008. (abstract, pdf)
   
   
3. Arto Klami, Craig Saunders, Teofilo de Campos, and Samuel Kaski Can relevance of images be inferred from eye movements?. MIR'08: Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval, Vancouver, British Columbia, Canada, Oct 30-31, 2008. (abstract, pdf)
   
   
4. Jaana Simola, Jarkko Salojärvi, and Ilpo Kojo: Using Hidden Markov Model to Uncover Processing States from Eye Movements in Information Search Tasks. Cognitive Systems Research 9:237-251, 2008. (DOI)

2007

 

1. David R. Hardoon, John Shawe-Taylor, Antti Ajanki, Kai Puolamäki, and Samuel Kaski: Information Retrieval by Inferring Implicit Queries from Eye Movements. Proceedings of the 11th International Conference on International Conference on Artificial Intelligence and Statistics, (AISTATS'07), 2007. (abstract, pdf)

2006

 

1. Kai Puolamäki and Samuel Kaski, editors. Proceedings of the NIPS 2005 Workshop on Machine Learning for Implicit Feedback and User Modeling. Helsinki University of Technology, Espoo, Finland, 2006.

2005

 

1. J. Salojärvi, K. Puolamäki, S. Kaski: On Discriminative Joint Density Modeling. In: Gama, Camacho, Brazdil, Jorge, Torgo (eds.): Machine Learning: ECML 2005. (Proceedings of 16th European Conference on Machine Learning), Lecture Notes in Artificial Intelligence 3270, pages 341-352. Springer-Verlag, Berlin, Germany. 2005. (abstract, pdf,DOI).
   
   
2. J. Salojärvi, K. Puolamäki, S. Kaski: Implicit Relevance Feedback from Eye Movements . In: Duch, Kacprzyk, Oja, Zadrozny (eds.): Artificial Neural Networks: Biological Inspirations - ICANN 2005: 15th International Conference. Proceedings, Part I. Lecture Notes in Computer Science 3696, pages 513 - 518. Springer-Verlag, Berlin, Germany. 2005. (abstract, pdf,DOI).
   
   
3. J. Salojärvi, K. Puolamäki, J. Simola, L. Kovanen, I. Kojo, S. Kaski: Inferring Relevance from Eye Movements: Feature Extraction . Helsinki University of Technology, Publications in Computer and Information Science, Report A82. 3 March 2005. (abstract, PDF)
   
   
4. K. Puolamäki, J. Salojärvi, E. Savia, J. Simola, S. Kaski: Combining Eye Movements and Collaborative Filtering for Proactive Information Retrieval .In: Marchionini, Moffat, Tait, Baeza-Yates, Ziviani (eds.): SIGIR 2005: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 146-153. ACM press, New York, USA. 2005.(abstract,pdf).
   
   
5. J. Salojärvi, K. Puolamäki, S. Kaski: Expectation Maximization Algorithms for Conditional Likelihoods . In Luc De Raedt and Stefan Wrobel, editors, Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), pp. 753-760. ACM press, New York, USA. 2005. (abstract, pdf).
   
   
6. E. Savia, K. Puolamäki, J. Sinkkonen, S. Kaski: Two-Way Latent Grouping Model for User Preference Prediction. In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI) 2005, pp. 518-525. (abstract, pdf).
   
   
7. S. Kaski: Proactive information retrieval by monitoring eye movements. In Abstracts of BrainIT, The Second International Conference on Brain-Inspired Information Technology, p. 28. Kyushu Institute of Technology, Kitakuyshu, Japan, 2005. Abstract of an invited talk. pdf.
   
   
8. S. Kaski, P. Myllymäki, and I. Kojo: User models from implicit feedback for proactive information retrieval. In C. de la Higuera and T. Artieres, editors, Proceedings of Workshop 4 of the 10th International Conference on User Modeling; Machine Learning for User Modeling: Challenges, pages 25-26. 2005. (abstract, pdf).
   
   

2004

 

1. Eerika Savia, Samuel Kaski, Ville Tuulos and Petri Myllymäki: On Text-Based Estimation of Document Relevance In Proceedings of the International Joint Conference on Neural Networks (IJCNN'04), Budapest, Hungary, July 2004, pp.3275-3280. (abstract, postscript, gzipped postscript, pdf).
   
   
2. Jarkko Salojärvi, Kai Puolamäki, and Samuel Kaski: Relevance feedback from eye movements for proactive information retrieval. In Workshop on Processing Sensory Information for Proactive Systems (PSIPS 2004), Oulu Finland, June 14-15, 2004. (abstract, postscript gzipped postscript, pdf).

   Also a poster presented at Machine Learning Meets the User Interface workshop at NIPS'03. (abstract[postscript],  poster[pdf]).

2003

 

1. Jarkko Salojärvi, Ilpo Kojo, Jaana Simola, and Samuel Kaski: Can relevance be inferred from eye movements in information retrieval? In Proceedings of the Workshop on Self-Organizing Maps (WSOM'03), Hibikino, Kitakyushu, Japan, September 2003. pp. 261-266. (abstract, postscript, gzipped postscript, pdf).

---