Recognition of the gaze direction: Anchoring with the eyebrows

•A multi stage fusion procedure for the localization of eye and eyebrow.•A remote gaze direction recognition system.•We show that using the eyebrow, better gaze precision is achievable especially on the vertical direction.

In this paper we investigate the accuracy of estimating a person’s direction of gaze from remote imaging. The problem is addressed by a person independent, multistage fusion approach for eye landmark localization, followed by eye region analysis for actual gaze recognition. We test the proposed landmark localization system on three databases, showing superior accuracy than state of the art solutions. Finally, we show that, inspired by human perception, by incorporating the location of eyebrows, superior performance is achievable when estimating the gaze direction. Given the found results, we argue that computer vision systems for gaze recognition should mimic the human perception and incorporate the eyebrows.