Recognition of 3D facial expression dynamics

In this paper we propose a method that exploits 3D motion-based features between frames of 3D facial geometry sequences for dynamic facial expression recognition. An expressive sequence is modelled to contain an onset followed by an apex and an offset. Feature selection methods are applied in order to extract features for each of the onset and offset segments of the expression. These features are then used to train GentleBoost classifiers and build a Hidden Markov Model in order to model the full temporal dynamics of the expression. The proposed fully automatic system was employed on the BU-4DFE database for distinguishing between the six universal expressions: Happy, Sad, Angry, Disgust, Surprise and Fear. Comparisons with a similar 2D system based on the motion extracted from facial intensity images was also performed. The attained results suggest that the use of the 3D information does indeed improve the recognition accuracy when compared to the 2D data in a fully automatic manner.

► This work proposes a novel method for dynamic 3D facial expression analysis. ► It employs motion-based features extracted through quad-tree decomposition. ► GentleBoost classifiers are used to recognise the onset/offset temporal segments. ► Hidden Markov Models are employed to model the full expression dynamics. ► Comparison with the 2D system demonstrated improvement with the 3D analysis.