Learning realistic facial expressions from web images

A large amount of labeled training data is required to develop effective and robust facial expression analysis methods. However, obtaining such data is typically a tedious and time-consuming task. With a rapid advance of the Internet and Web technologies, it has been feasible to collect a large number of images with label information at a low cost of human efforts. In this paper, we propose a search based framework to collect realistic facial expression images from the Web so as to further advance research on robust facial expression recognition. Due to the limitation of current commercial web search engines, a large fraction of returned images is not related to a given query keyword. We present a Support Vector Machine (SVM) based active learning approach for selecting relevant images from noisy image search results. The resulting dataset is more diverse with more sample images per expression compared to other well established facial expression datasets such as CK and JAFFE. In addition, a novel facial expression feature based on the state-of-the-art Weber Local Descriptor (WLD) and histogram contextualization is proposed to handle such a challenging dataset. Comprehensive experimental results demonstrate that our web based dataset is capable of resembling more closely to the real world conditions compared to the CK and JAFFE datasets, and our proposed feature is more effective than the existing widely used features.

► We propose a semi-automatic framework to build a facial expression database.
► A novel face representation is presented for multi-resolution analysis of faces.
► The images in our database can more closely resemble the real world environments.
► The proposed face representation is more robust under challenging conditions.