Unsupervised visual hull extraction in space, time and light domains

• Background likelihood estimation based on temporal pixel intensity analysis.
• A new type of filtering that preserves edges in a signal.
• Information fusion from different domains and its integration into an MRF.
• An implementation framework capable of producing VH for highly challenging situations.

This paper presents an unsupervised image segmentation approach for obtaining a set of silhouettes along with the visual hull (VH) of an object observed from multiple viewpoints. The proposed approach can deal with mostly any type of appearance characteristics such as texture, similar background color, shininess, transparency besides other phenomena such as shadows and color bleeding. Compared to more classical methods for silhouette extraction from multiple views, for which certain assumptions are made on the object or scene, neither the background nor the object appearance properties are modeled. The only assumption is the constancy of the unknown background for a given camera viewpoint while the object is under motion. The principal idea of the method is the estimation of the temporal evolution of each pixel over time which provides a stability measurement and leads to its associated background likelihood. In order to cope with shadows and self-shadows, an object is captured under different lighting conditions. Furthermore, the information from the space, time and lighting domains is exploited and merged based on a MRF framework and the constructed energy function is minimized via graph cut. Experiments are performed on a light stage where the object is set on a turntable and is observed from calibrated viewpoints on a hemisphere around the object. Real data experiments show that the proposed approach allows for robust and efficient VH reconstruction of a variety of challenging objects.