Face image classification by pooling raw features

We propose a very simple, efficient yet surprisingly effective feature extraction method for face recognition (about 20 lines of Matlab codes), which is mainly inspired by spatial pyramid pooling in generic image classification. We show that, coupled with a linear classifier, features formed by simply pooling local patches over a multi-level pyramid can achieve state-of-the-art performance on face recognition. The simplicity of our feature extraction procedure is demonstrated by the fact that no learning is involved (except PCA whitening). It is shown that multi-level spatial pooling and dense extraction of multi-scale patches play critical roles in face image classification. The extracted facial features can capture strong structural information of individual faces with no label information being used. We also find that pre-processing on local image patches such as contrast normalization can have an important impact on the classification accuracy. In particular, on the challenging face recognition datasets of FERET and LFW-a, our method improves previous best results by large gaps. Promising results are also achieved on the general image classification database Caltech-101.