A general framework for three-dimensional surface reconstruction by self-consistent fusion of shading and shadow features

In this paper a novel framework for three-dimensional surface reconstruction by self-consistent fusion of shading and shadow features is presented. Based on the analysis of at least two pixel-synchronous images of the scene under different illumination conditions, this framework combines a shape from shading approach for estimating surface gradients and altitude variations on small scales with a shadow analysis method that allows for the determination of the large-scale properties of the surface. As a first step, the result of shadow analysis is used for selecting a consistent solution of the shape from shading reconstruction algorithm. As a second step, an additional error term derived from the fine-structure of the shadow is incorporated into the reconstruction algorithm. This approach is extended to the analysis of two or more shadows under different illumination conditions leading to an appropriate initialization of the shape from shading algorithm. The framework is applied to the astrogeological task of three-dimensional reconstruction of regions on the lunar surface using ground-based CCD images and to the machine vision task of industrial quality inspection.