Facial expression recognition using tracked facial actions: Classifier performance analysis

In this paper, we address the analysis and recognition of facial expressions in continuous videos. More precisely, we study classifiers performance that exploit head pose independent temporal facial action parameters. These are provided by an appearance-based 3D face tracker that simultaneously provides the 3D head pose and facial actions. The use of such tracker makes the recognition pose- and texture-independent. Two different schemes are studied. The first scheme adopts a dynamic time warping technique for recognizing expressions where training data are given by temporal signatures associated with different universal facial expressions. The second scheme models temporal signatures associated with facial actions with fixed length feature vectors (observations), and uses some machine learning algorithms in order to recognize the displayed expression. Experiments quantified the performance of different schemes. These were carried out on CMU video sequences and home-made video sequences. The results show that the use of dimension reduction techniques on the extracted time series can improve the classification performance. Moreover, these experiments show that the best recognition rate can be above 90%.

► We address the analysis and recognition of facial expressions in continuous videos.
► We study classifier performance for pose- and texture-independent approaches.
► The first proposed approach is based on elastic matching of temporal signatures.
► The second proposed approach is based on a linear embedding technique.
► Recognition performance is provided for a CMU face subset and home-made videos.