Shape-based image segmentation through photometric stereo

This paper describes a new algorithm for segmenting 2D images by taking into account 3D shape information. The proposed approach consists of two stages. In the first stage, the 3D surface normals of the objects present in the scene are estimated through robust photometric stereo. Then, the image is segmented by grouping its pixels according to their estimated normals through graph-based clustering. One of the advantages of the proposed approach is that, although the segmentation is based on the 3D shape of the objects, the photometric stereo stage used to estimate the 3D normals only requires a set of 2D images. This paper provides an extensive validation of the proposed approach by comparing it with several image segmentation algorithms. Particularly, it is compared with both appearance-based image segmentation algorithms and shape-based ones. Experimental results confirm that the latter are more suitable when the objective is to segment the objects or surfaces present in the scene. Moreover, results show that the proposed approach yields the best image segmentation in most of the cases.

Research highlights
► Surface normals and reflectance are recovered from 2D information.
► Pixels with similar 3D surface normals are clustered together.
► Shape-based image segmentation is obtained from only six gray-level intensity images.
► Examples with low gray-level intensity variation and noisy background pixels are provided.
► Extensive evaluation of the proposed approach is performed.