Multi-gradient features and elongated quinary pattern encoding for image-based facial expression recognition

In this paper we propose a novel texture feature extraction method for posed and spontaneous image based facial expression recognition. The kernel Sobel filter is used with eight masks to derive the gradient components for each pixel in the image. Two types of gradient images are extracted for different directions denoted as xy and lr. The robust Elongated Quinary Pattern (EQP) descriptor is then used to quantize neighborhood local gradients around each point using five discrimination levels. We next divide each encoded image into a number of blocks and concatenate the local histogram features of each image individually. In order to boost the performance, we adopt a Multi Classifier System (MCS) to combine all scores of the encoded images based upon a multi-class Support Vector Machine (SVM) classifier. Experimental results show a significant improvement over previous approaches in the average recognition accuracy when using the spontaneous Moving Faces and People (MFP) database. In addition, the proposed method outperformed state-of-the-art methods when applied to the posed CK database with a recognition performance of 99.36% in the case of seven classes and 99.72% without the neutral class.