Face alignment by robust discriminative Hough voting

• Extending the range of traditional CLM framework by including the nonparametric (exemplar-based) shape models.
• Incorporating the prior knowledge of very few anchor points into the task of face alignment, while being robust against localization errors of anchor points.
• Exploiting the power of discriminative learning under the generative exemplar-based CLM.
• Promising results on four challenging databases, i.e., LFW, LFPW, HELEN and IBUG.
• Exploring the deep CNN technique in the RANSAC step to further boost the performance of the proposed system.

This paper presents a novel Hough voting-based approach for face alignment under the extended exemplar-based Constrained Local Models (CLMs) framework. The main idea of the proposed method is to use very few stable facial points, i.e., anchor points, to help reduce the ambiguity encountered when localizing other less stable facial points by Hough voting. A less studied limitation of Hough voting-based methods, however, is that their performance is typically sensitive to the quality of anchor points, especially when only very few (e.g., one pair of) anchor points are used. In this paper, we mainly focus on this issue and our major contributions are three-fold: (1) We first propose a novel method to evaluate the goodness of anchor points based on the diagnosis of resulted distribution of their votings for other facial points; (2) To deal with the remaining small localization errors, an enhanced RANSAC method is presented, in which a sampling strategy is adopted to soften the range of possible locations of the chosen anchor points, and the top ranking exemplars are then selected based on a newly-proposed cost-sensitive discriminative objective; (3) Finally, both global voting priors and local evidence are fused under a weighted least square framework. Experiments on several challenging datasets, including LFW, LFPW, HELEN and IBUG, demonstrate that the proposed method outperforms many state-of-the-art CLM methods. We also show that the performance of the proposed system can be further boosted by exploring the deep CNN technique in the RANSAC step.