Lucas–Kanade based entropy congealing for joint face alignment

Entropy Congealing is an unsupervised joint image alignment method, in which the transformation parameters are obtained by minimizing a sum-of-entropy function. Our previous work presented a forward formulation of entropy Congealing to estimate all the transformation parameters at the same time. In this paper, we propose an inverse compositional Lucas–Kanade formulation of entropy Congealing. This yields constant parts in Jacobian and Hessian which can be precomputed to decrease the computational complexity. Moreover, we combine Congealing with POEM descriptor to catch more information about face. Experimental results indicate that the proposed algorithm performs better than other alignment methods, regarding several evaluation criteria on different databases. Concerning the complexity, the proposed algorithm is more efficient than other considered approaches. Also, compared to the forward formulation, the inverse method produces a speed improvement of 20%.

► Presented in Section 2 are the canonical Congealing method and POEM descriptor.
► Section 3 presents new Entropy Congealing methods using Lucas–Kanade formulations.
► Main steps of forward and inverse LKC algorithms are respectively shown in Figs. 2 and 3.
► Our algorithm works well under different image conditions (Figs. 10–18).
► Our alignment method improves face recognition performance (Figs. 19 and 20, Table 2).
► Table 3 proves the computational efficiency of the proposed algorithm.