Translational photometric alignment of single-view image sequences

Photometric stereo is a well-established method to estimate surface normals of an object. When coupled with depth-map estimation, it can be used to reconstruct an object’s height field. Typically, photometric stereo requires an image sequence of an object under the same viewpoint but with differing illumination directions. One crucial assumption of this configuration is perfect pixel correspondence across images in the sequence. While this assumption is often satisfied, certain setups are susceptible to translational errors or misalignments across images. Current methods to align image sequences were not designed specifically for single-view photometric stereo. Thus, they either struggle to account for changing illumination across images, require training sets, or are overly complex for these conditions. However, the unique nature of single-view photometric stereo allows one to model misaligned image sequences using the underlying image formation model and a set of translational shifts. This paper introduces such a technique, entitled translational photometric alignment, that employs the Lambertian model of image formation. This reduces the alignment problem to minimizing a nonlinear sum-squared error function in order to best reconcile the observed images with the generative model. Thus, the end goal of translational photometric alignment is not only to align image sequences, but also to produce the best surface-normal estimates given the observed images. Controlled experiments on the Yale Face Database B demonstrate the high accuracy of translational photometric alignment. The utility and benefits of the technique are further illustrated by additional experiments on image sequences suffering from uncontrolled real-world misalignments.

► Translational mismatch is modelled using image formation equation.
► Misalignments are corrected using nonlinear least squares.
► Structure of problem allows efficient gradient and Hessian computation.
► Severely misaligned and challenging image sequences are aligned with high accuracy.
► Tangible benefits are provided to a real-world scenario.