Non-rigid visible and infrared face registration via regularized Gaussian fields criterion

• We analyze the robustness of Gaussian fields criterion both in theory and experiment.
• The Gaussian fields criterion is generalized from rigid to the non-rigid case.
• We propose a new non-rigid registration method to deal with more real-world problems.
• A sparse approximation on the transformation is applied to speed up the method.
• We customize and apply the proposed method to visible/thermal IR face registration.

Registration of multi-sensor data (particularly visible color sensors and infrared sensors) is a prerequisite for multimodal image analysis such as image fusion. Typically, the relationships between image pairs are modeled by rigid or affine transformations. However, this cannot produce accurate alignments when the scenes are not planar, for example, face images. In this paper, we propose a regularized Gaussian fields criterion for non-rigid registration of visible and infrared face images. The key idea is to represent an image by its edge map and align the edge maps by a robust criterion with a non-rigid model. We model the transformation between images in a reproducing kernel Hilbert space and a sparse approximation is applied to the transformation to avoid high computational complexity. Moreover, a coarse-to-fine strategy by applying deterministic annealing is used to overcome local convergence problems. The qualitative and quantitative comparisons on two publicly available databases demonstrate that our method significantly outperforms the state-of-the-art method with an affine model. As a result, our method will be beneficial for fusion-based face recognition.