Face alignment recurrent network

This paper presents a new facial landmark detection method for images and videos under uncontrolled conditions, based on a proposed Face Alignment Recurrent Network (FARN). The network works in recurrent fashion and is end-to-end trained to help avoid over-strong early stage regressors and over-weak later stage regressors as in many existing works. Long Short Term Memory (LSTM) model is employed in our network to make full use of the spatial and temporal middle stage information in a natural way, where by spatial we mean that for each image (frame), the predicted landmark position in the current stage will be used to guide the estimation for the next stage, and by temporal we mean that the predicted landmark position in the current frame will be used to guide the estimation for the next frame, and thus providing an unified framework for facial landmark detection in both images and videos. We conduct experiments on public image datasets (COFW, Helen, 300-W) as well as on video datasets (300-VW), and results show clear improvement over most of the current state-of-the-art approaches. In addition, it works in 18 ms per image (frame).1