Feature fusion for facial landmark detection

•Novel generalized framework for combining facial landmark descriptors.•The combination of landmark descriptors is an under-studied issue.•Several feature fusion schemes for landmark detection are proposed and evaluated.•The proposed approach offers dimensionality reduction and is easily extendable.•The proposed approach can combine features extracted from 3D and 2D facial data.

Facial landmark detection is a crucial first step in facial analysis for biometrics and numerous other applications. However, it has proved to be a very challenging task due to the numerous sources of variation in 2D and 3D facial data. Although landmark detection based on descriptors of the 2D and 3D appearance of the face has been extensively studied, the fusion of such feature descriptors is a relatively under-studied issue. In this paper, a novel generalized framework for combining facial feature descriptors is presented, and several feature fusion schemes are proposed and evaluated. The proposed framework maps each feature into a similarity score and combines the individual similarity scores into a resultant score, used to select the optimal solution for a queried landmark. The evaluation of the proposed fusion schemes for facial landmark detection clearly indicates that a quadratic distance to similarity mapping in conjunction with a root mean square rule for similarity fusion achieves the best performance in accuracy, efficiency, robustness and monotonicity.

Graphical abstractFigure optionsDownload full-size imageDownload high-quality image (90 K)Download as PowerPoint slide