Facial expression recognition from near-infrared videos

Facial expression recognition is to determine the emotional state of the face regardless of its identity. Most of the existing datasets for facial expressions are captured in a visible light spectrum. However, the visible light (VIS) can change with time and location, causing significant variations in appearance and texture. In this paper, we present a novel research on a dynamic facial expression recognition, using near-infrared (NIR) video sequences and LBP-TOP (Local binary patterns from three orthogonal planes) feature descriptors. NIR imaging combined with LBP-TOP features provide an illumination invariant description of face video sequences. Appearance and motion features in slices are used for expression classification, and for this, discriminative weights are learned from training examples. Furthermore, component-based facial features are presented to combine geometric and appearance information, providing an effective way for representing the facial expressions. Experimental results of facial expression recognition using a novel Oulu-CASIA NIR&VIS facial expression database, a support vector machine and sparse representation classifiers show good and robust results against illumination variations. This provides a baseline for future research on NIR-based facial expression recognition.

Figure optionsDownload high-quality image (241 K)Download as PowerPoint slideHighlights
► Combination of NIR imaging and LBP-TOP features for an illumination invariant system.
► Weight assignment to learn most discriminative appearance and motion slices.
► Component-based representations for dealing with pose changes and partial occlusion.
► Novel NIR and VIS facial expressions database and baseline results.