Learning spatial weighting for facial expression analysis via constrained quadratic programming

•A novel optimization formulation for facial expression analysis is proposed.•This formulation learns spatial weighting for optical flow due to facial expression.•This learning formulation leads to solving a quadratic programming problem.•Facial landmarks are first located, and optical flow is computed as the motion features.•Accuracy is greatly improved on expression recognition and intensity estimation experiments.

Facial expression analysis is essential for human–computer interface. For different expressions, different parts of the face play different roles due to distinct movement of facial muscles. In this work, we propose to learn the weight associated with different facial regions for different expressions. The facial feature points are first located accurately based on a graphical model. Based on using the optical flow to represent the motion information due to facial expression, a quadratic programming problem is formulated to learn the optimal spatial weighting from training data such that faces of the same expression category are closer than those of different categories in the weighted optical flow space. We demonstrate the advantages of applying the learned weight to facial expression recognition and intensity estimation through experiments on several well-known facial expression databases.