Multiview Distance Metric Learning on facial feature descriptors for automatic pain intensity detection

•Appearance based feature descriptors have been used for pain intensity detection on a frame by frame basis.•To map the data into higher discriminative space and to extract complimentary information from them, Multiview Distance Metric learning (MDML) technique is used.•SVM classifier gives better results for MDML extracted features rather than other dimension reduction methods.•The pain detection accuracy achieved is comparable to the current popular methods for detection of pain and pain intensity.

In this study, a novel approach for automatic pain intensity detection is presented that capitalizes the complimentary information from various facial feature descriptors. We extract facial features by using the popular feature descriptors, which include Gabor features, Histogram of Orientation Gradient features and Local Binary Pattern features. A Multiview Distance Metric Learning (MDML) method is applied on these features to seek a common distance metric such that the features of the frames belonging to the same pain intensity are pulled as close as possible and that belonging to the different intensity levels are pushed as far as possible, simultaneously. Moreover, MDML extracts complementary information from various feature descriptors. The feature vector so obtained, are applied to Support Vector Machine for pain detection and pain intensity detection. We assess our algorithm on UNBC-McMaster Shoulder Pain Expression Archive Database. Experimental results represent that the efficiency of the proposed approach is 90% for pain detection and 75% for four level pain detection, which represents the potential of the proposed approach. A comparison of the proposed MDML approach is presented with other popular dimension reduction methods to prove its feature extraction capability.