Multimodal Interfaces

Our research in the Multimodal Interfaces group aims at solving the problem of information explosion by fusing several of the available perceptual and user feedback modalities. The results will be applied in interactive interfaces, for example in content-based multimodal retrieval.

The inter-modal cross-over of semantics such as image, audio and text segments corresponding to each other can be accomplished, e.g., by a so-called point-and-tell interface like the one above.

The Multimodal Interfaces group coordinates our work on relevant subtopics, namely image retrieval, speech recognition, proactive information retrieval, and natural language processing.

Contact: Mikko Kurimo, Mikko.Kurimo at hut.fi