Learning from weakly labeled faces and video in the wild

• A probabilistic sparse kernel technique able to learn with noisy labels.
• Method consistently boosts performance on real world face recognition tasks such as the Labeled Faces in the Wild evaluation.
• Experiments show the importance of using non-linear classifiers when data is weakly labeled.
• The method can learn from weakly annotated video where the subject׳s face is not always present.

We present a novel method for face-recognition based on leveraging weak or noisily labeled data. We combine facial images from the Labeled Faces in the Wild (LFW) dataset with face images extracted from videos on YouTube and face images returned using a search engine. Our technique is based on a novel formulation for weakly supervised learning based on probabilistic graphical models using a margin-like property and a null category. As such, our formulation remains within a fully probabilistic framework. We use this technique to combine high accuracy human labeled data with noisily labeled data. We present a specific variation of our general approach using a model inspired by the relevance vector machine (RVM), a probabilistic alternative to support vector machines. In contrast to previous formulations of RVMs we show how the choice of an exponential hyperprior produces an approximation to the L1 penalty. We present both experiments where we simulate noisy labels and experiments where we use image and video search results as noisily labeled data. Faces extracted from the resulting Youtube videos thus are likely, but not assured to contain examples of the person whose name was given as the query. We show how our probabilistic margin approach provides a robust way to combine labeled LFW data with this type of noisy search result. Our results indicate that recognition performance can indeed be increased consistently with weakly labeled data using our technique.