Vector transport for shape-from-shading

In this paper we describe a new shape-from-shading method. We show how the parallel transport of surface normals can be used to impose curvature consistency and also to iteratively update surface normal directions so as to improve the brightness error. We commence by showing how to make local estimates of the Hessian matrix from surface normal information. With the local Hessian matrix to hand, we develop an “EM-like” algorithm for updating the surface normal directions. At each image location, parallel transport is applied to the neighbouring surface normals to generate a sample of local surface orientation predictions. From this sample, a local weighted estimate of the image brightness is made. The transported surface normal which gives the brightness prediction which is closest to this value is selected as the revised estimate of surface orientation. The revised surface normals obtained in this way may in turn be used to re-estimate the Hessian matrix, and the process iterated until stability is reached. We experiment with the method on a variety of real world and synthetic data. Here we explore the properties of the fields of surface normals and the height data delivered by the method.