Focus-aided scene segmentation

• Focus has been typically exploited in computer vision as depth cue.
• A new focus-based perceptual cue is introduced: the focus signal.
• The focus signal corresponds to the change in focus level as a function of time.
• The focus signal can be integrated with classical cues for image segmentation.
• The proposed focus-aided methodology yields improved scene segmentation.

Classical image segmentation techniques in computer vision exploit visual cues such as image edges, lines, color and texture. Due to the complexity of real scenarios, the main challenge is achieving meaningful segmentation of the imaged scene since real objects have substantial discontinuities in these visual cues. In this paper, a new focus-based perceptual cue is introduced: the focus signal. The focus signal captures the variations of the focus level of every image pixel as a function of time and is directly related to the geometry of the scene. In a practical application, a sequence of images corresponding to an autofocus sequence is processed in order to infer geometric information of the imaged scene using the focus signal. This information is integrated with the segmentation obtained using classical cues, such as color and texture, in order to yield an improved scene segmentation. Experiments have been performed using different off-the-shelf cameras including a webcam, a compact digital photography camera and a surveillance camera. Obtained results using Dice’s similarity coefficient and the pixel labeling error show that a significant improvement in the final segmentation can be achieved by incorporating the information obtained from the focus signal in the segmentation process.