Activity recognition using eye-gaze movements and traditional interactions

The need for intelligent HCI has been reinforced by the increasing numbers of human-centered applications in our daily life. However, in order to respond adequately, intelligent applications must first interpret users’ actions. Identifying the context in which users’ interactions occur is an important step toward automatic interpretation of behavior. In order to address a part of this context-sensing problem, we propose a generic and application-independent framework for activity recognition of users interacting with a computer interface. Our approach uses Layered Hidden Markov Models (LHMM) and is based on eye-gaze movements along with keyboard and mouse interactions. The main contribution of the proposed framework is the ability to relate users’ interactions to a task model in variant applications and for different monitoring purposes. Experimental results from two user studies show that our activity recognition technique is able to achieve good predictive accuracy with a relatively small amount of training data.

► Eye-gaze movements require noize tolerant classification algorithms. ► Activity recognition framework is generic and application-independant. ► Good prediction accuracy with few training data.