Automatic alignment and reconstruction of facial depth images

•An automatic technique for aligning and reconstructing damaged facial depth images.•A block division approach that allows the use of a robust interpolation scheme.•A relief mapping-based ray casting strategy to resample the reconstructed surface.•A comparative study between conventional 2D/3D and the proposed 3D alignment.•A comparative study of the performance of four different interpolation methods.

Face, gender, ethnic and age group classification systems often work through an alignment, feature extraction, and identification pipeline. The quality of the alignment process is thus central to the performance of the identification process. Furthermore, missing portions of depth information can greatly affect results. Appropriate image reconstruction is therefore crucial for the correct operation of those systems. This paper presents a simple and effective approach for the automatic alignment and reconstruction of damaged facial depth images. By using only four facial landmarks and the raw depth data, our approach converts a given damaged depth image into a smooth depth function, performs the 3D alignment of the underlying face with the face of an average person, and produces an aligned depth image having arbitrary resolution. Our experiments show that the proposed approach outperforms commonly used methods. For instance, we show that it improves the quality of a state-of-art gender classification technique.