Dynamic soft encoded patterns for facial event analysis

In this paper, we propose a new feature: dynamic soft encoded pattern (DSEP) for facial event analysis. We first develop similarity features to describe complicated variations of facial appearance, which take similarities between a haar-like feature in a given image and the corresponding ones in reference images as feature vector. The reference images are selected from the apex images of facial expressions, and the k-means clustering is applied to the references. We further perform a temporal clustering on the similarity features to produce several temporal patterns along the temporal domain, and then we map the similarity features into DSEP to describe the dynamics of facial expressions, as well as to handle the issue of time resolution. Finally, boosting-based classifier is designed based on DSEPs. Different from previous works, the proposed method makes no assumption on the time resolution. The effectiveness is demonstrated by extensive experiments on the Cohn–Kanade database.

Research highlights► The impaction of the time resolution on expression recognition is analyzed. ► We proposed the dynamic encoded pattern to handle time resolution problem. ► Dynamic Soft Encoded Patterns are used to further improve the performance.