Boosted NNE collections for multicultural facial expression recognition

•Happy, surprise are universal expressions, easy to classify on multicultural dataset.•Concentration region extraction offer consistent mapping in multicultural expression.•Uniform visual representation will be the future work in this area.•Boosted NNE collections performed well using HOG and PCA features.

In this paper, a boosted NNE (neural network ensemble) collections based technique for multicultural facial expression recognition is presented. The boosted NNE collections based ensemble classifier involves three steps: first is the training of binary neural networks, second is to combine the predictions of binary neural networks to form NNE, and third is to combine the predictions of NNE collections in order to detect the presence of an expression. The results of binary neural networks are tied to the probability value across the NNE collection. The boosting technique is applied for the construction of NNEs and the final prediction is made by Naive Bayes classifier. The Voila–Jones algorithm [1] employed for the detection of face and facial expression concentration region. The acted still images from three databases JAFFE, TFEID, and RadBoud originate from four different cultural and ethnic regions including Japanese, Taiwanese, Caucasians and Moroccans are combined to develop the cross cultural facial expression dataset. This cross cultural facial expression dataset is used for the training and testing of binary neural networks in each NNE collection. Three different feature extraction techniques PCA, LBP and HOG are used for sample image representation. The experimental results and statistical analysis of proposed approach for multicultural facial expression recognition constitute the contribution to the field.