A graphical model based solution to the facial feature point tracking problem

In this paper a facial feature point tracker that is motivated by applications such as human–computer interfaces and facial expression analysis systems is proposed. The proposed tracker is based on a graphical model framework. The facial features are tracked through video streams by incorporating statistical relations in time as well as spatial relations between feature points. By exploiting the spatial relationships between feature points, the proposed method provides robustness in real-world conditions such as arbitrary head movements and occlusions. A Gabor feature-based occlusion detector is developed and used to handle occlusions. The performance of the proposed tracker has been evaluated on real video data under various conditions including occluded facial gestures and head movements. It is also compared to two popular methods, one based on Kalman filtering exploiting temporal relations, and the other based on active appearance models (AAM). Improvements provided by the proposed approach are demonstrated through both visual displays and quantitative analysis.

Graphical AbstractFigure optionsDownload full-size imageDownload high-quality image (74 K)Download as PowerPoint slideResearch Highlights►A graphical model based approach for facial feature tracking. ►Temporal and spatial constraints are utilized for 3D head movements and occlusions. ►Comparison with Kalman filtering and AAM both qualitatively and quantitatively. ►Although there are some drifts, our tracker appears to be a better than the others.