Facial landmark detection by semi-supervised deep learning

In this paper, we propose a semi-supervised facial landmark detection algorithm (SEMI) based on convolutional neural network (CNN), which can detect facial components and landmarks simultaneously. Unlike previous coarse-to-fine algorithms, our model does not need extra input such as initial landmark prediction. It also solves the occlusion problem of large area by detecting the visible facial components while existing face detectors failed to detect faces. Semi-supervised learning algorithm is also an effective data augmentation method. In our experiment, each image has two types of ground truth, one is bounding-box related (classification and coordinates) and the other is landmark coordinates inside the bounding-box. The supervised data have both two types of ground truth while the semi-supervised data only have the bounding-box. Our model was trained by the merge of two parts of data. Extensive evaluations on Helen, LFPW and 300-W show that our algorithm is able to complete the landmark task and performs better than many state-of-the-art facial landmark detecting algorithms.