Capturing global spatial patterns for distinguishing posed and spontaneous expressions

• A method to differentiate posed and spontaneous expressions by capturing their global spatial patterns using RBM is proposed.
• Partition function of RBM is estimated by extending AIS to RBM with continuous visible units.
• Gender and expression category applied as privileged information for posed and spontaneous expression distinction.
• Experimental results on benchmark databases demonstrate the effectiveness of the proposed approach.

In this paper, we introduce methods to differentiate posed expressions from spontaneous ones by capturing global spatial patterns embedded in posed and spontaneous expressions, and by incorporating gender and expression categories as privileged information during spatial pattern modeling. Specifically, we construct multiple restricted Boltzmann machines (RBMs) with continuous visible units to model spatial patterns from facial geometric features given expression-related factors, i.e., gender and expression categories. During testing, only facial geometric features are provided, and the samples are classified into posed or spontaneous expressions according to the RBM with the largest likelihood. Furthermore, we propose efficient inference algorithm by extending annealing importance sampling to RBM with continuous visible units for calculating partition function of RBMs. Experimental results on benchmark databases demonstrate the effectiveness of the proposed approach in modelling global spatial patterns as well as its superior posed and spontaneous expression distinction performance over existing approaches.