Differential components of discriminative 2D Gaussian–Hermite moments for recognition of facial expressions

• A novel facial expression recognition algorithm using 2D Gaussian–Hermite moments.
• New approach of discriminative selection of moments as features of expression.
• New subspace to estimate differential components of moments as expressive features.
• Experiments on challenging datasets having posed, spontaneous, and wild expressions.
• Results show that proposed method is better than existing or similar methods.

This paper deals with a new expression recognition method by representing facial images in terms of higher-order two-dimensional orthogonal Gaussian–Hermite moments (GHMs) and their geometric invariants. Only the moments having high discrimination power are selected as a set of features for expressions. To obtain the differentially expressive components of the moments, the discriminative GHMs are projected on to a new expression-invariant subspace using the correlations among the neutral faces. Features obtained from the discriminative moments and differentially expressive components of the moments are used to recognize an expression using the well-known support vector machine classifier. Experimental results presented are obtained from commonly-referred databases such as the CK-AUC, FRGC, and MMI that have posed or spontaneous expressions as well as the GENKI database that has expressions in-the-wild. Experiments on mutually exclusive subjects reveal that the performance of expression recognition of the proposed method is significantly better than that of the existing or similar methods, which use the local or patch-based high dimensional binary patterns, directional number patterns generated from derivatives of Gaussian, Gabor- or other moment-based features.