An ensemble of patch-based subspaces for makeup-robust face recognition

•We propose a face-matching framework to address the problem of makeup.•An ensemble of subspaces is used for face representation and matching.•Each subspace pertains to a set of randomly selected patches.•Multiple texture descriptors are used to describe a patch.•Combination of sparse and collaborative classifiers is used in these subspaces.•Large number of experiments are conducted to convey the efficacy of the scheme.

Recent research has demonstrated the negative impact of makeup on automated face recognition. In this work, we introduce a patch-based ensemble learning method, which uses multiple subspaces generated by sampling patches from before-makeup and after-makeup face images, to address this problem. In the proposed scheme, each face image is tessellated into patches and each patch is represented by a set of feature descriptors, viz., Local Gradient Gabor Pattern (LGGP), Histogram of Gabor Ordinal Ratio Measures (HGORM) and Densely Sampled Local Binary Pattern (DS-LBP). Then, an improved Random Subspace Linear Discriminant Analysis (SRS-LDA) method is used to perform ensemble learning by sampling patches and constructing multiple common subspaces between before-makeup and after-makeup facial images. Finally, Collaborative-based and Sparse-based Representation Classifiers are used to compare feature vectors in this subspace and the resulting scores are combined via the sum-rule. The proposed face matching algorithm is evaluated on the YMU makeup dataset and is shown to achieve very good results. It outperforms other methods designed specifically for the makeup problem.