Automatic face interpretation using fast 3D illumination-based AAM models

We present an innovative and fast approach for face interpretation invariant to lighting and pose. Our approach performs interpretation by fitting a parametric 3D face model to an input image using an optimization algorithm. The parameters obtained after the fitting process describe the appearance of the face. The fitting process is automatic and only requires a 2D position and a scale factor as initialization. The proposed model is a natural 3D extension of active appearance models and is based on modeling, separately and simultaneously, 3D pose, 3D shape, albedo, and lighting. Our model is capable of synthesizing faces with arbitrary 3D shape, 3D pose, albedo and lighting. In order to fit the model to an input image, we propose a fast optimization algorithm able to fit face images with non-uniform lighting and arbitrary pose. Our algorithm, based on a gradient descent approach, executes a fast update to the Jacobian by using the lighting parameters estimated in each iteration of the fitting process. We show that our method is able to accurately estimate the parameters of 3D shape and albedo, which are strongly related to identity. Experimental results, suggest that our model can be extended to face recognition under non-uniform lighting and variable pose.

Research highlights
► Automatic and fast pose and lighting-invariant interpretation of faces is possible.
► AAMs can manage lighting variation by adapting a dense model of surface normals.
► A fast fitting algorithm can be implemented by using a lighting-adaptive Jacobian.