ISL eye gaze tracking

While the orientation of the head determines the overall direction of the gaze, the orientation of the eyes determines the exact gaze direction and is limited by the head orientation. In this project, we focus on monitoring a user's eye gaze, that is, estimating what a user is looking at on a screen based on information of the user's eye gaze. The research project on estimating head orientation can be found on our web page "Model-based Gaze Tracking". The two projects will be combined in the future to obtain general gaze information.

A good eye tracker is a prerequisite of eye gaze monitoring.
Many eye gaze tracking methods rely on intrusive techniques such as measuring the reflection of some light (usually infrared light) that is shone onto the eye, measuring the electric potential of the skin around the eyes or applying special contact lenses that facilitate eye gaze tracking.
This causes serious problems of user acceptance. We have developed a non-intrusive facial feature tracker that can detect and track a user's eyes in real-time (15+ Hz) as soon as the face appears in the view of the camera without the need of any special lighting or any marks on the user's face.

A user's eye gaze can be estimated by eye images. Baluja and Pomerleau have demonstrated that a neural network could accurately estimate the position of the eye gaze on a computer screen given images of the user's eyes as input ( see TR), though their system used an active sensing approach by shining light into the user's right eye which causes a problem of user acceptance. We have built an ANN based eye gaze monitoring system that uses our eye tracker to find and extract the eyes in real time. Preprocessed images of the user's eyes are then fed into the neural net which estimates the location of the user's eye gaze on a computer screen.

The eye gaze monitoring system has achieved accuracy between 1.3 degrees and 1.8 degrees for user dependent neural networks and accuracy of 1.9 degrees for a multiuser network.

User dependent accuracy (degrees)
preprocessing	User 1	User 2	User 3	User 4
histogram norm.	1.4	1.5	1.8	2.0
+ templ. matching	1.5	1.3	1.6	1.8

Multi-user accuracy (degrees)
preprocessing User 1,2,3,4

histogram norm. 2.3

+ templ. matching 1.9

One of the problems in the current gaze tracking system is that only local information, i.e., the images of the eyes, is used for estimating the user's gaze. Consequently the system relies on a relatively stable position of the users head with respect to the camera and the user should not rotate his head.

To make the gaze tracking system more robust to user movement, it would be helpful to also use additional information such as the 3D position of the head relative to the camera to estimate the users gaze.
We are currently working on combining the user's eye gaze information with information about the user's 3D head position and orientation, which we can already automatically track using our head pose tracker .

In the current system, the problem of deriving the focus of attention from the user's 'low level' eye gaze patterns has not yet been addressed.
In fact, even if we could have a perfect gaze tracking system, we still have the problem of finding a user's focus of attention using only gaze information. A high-evel user model is needed to deal with involuntary eye movements. We will address this problem in our further research.

Related publications:

Tracking Eyes and Monitoring Eye Gaze
Rainer Stiefelhagen, Jie Yang, Alex Waibel
Workshop on Perceptual User Interfaces, Banff, Canada
(postscript, 650 kb)

KEYWORDS Eye Tracking, Gaze Tracking, Focus of Attention