The purpose of the INTERACT project is to enhance human-computer communication by the processing and combination of multiple communication modalities known to be helpful in human communicative situations.
One of our goals is to derive a better model of where a person is in a room, who he/she might be talking to, and what he/she is saying despite the presence of jamming speakers and sounds in the room (the cocktail party effect).

We are also working to interpret the joint meaning of gestures and handwriting in conjunction with speech, so that computer applications ("assistants") can carry out actions more robustly and naturally and in more flexible ways. Several human-computer interaction tasks are explored to see how automatic gesture, speech and handwriting recognition, face and eye tracking, lipreading and sound source localization can all help to make human-computer interaction easier and more natural.

The INTERACT project: