Facet-based multiple building analysis for robot intelligence

This paper describes an approach to segment and recognize multiple buildings in the urban environment for robot intelligence. By grouping line segments which coincide with a common vanishing point, the non-building and building images are distinguished. The facets of building are detected and represented by the meshes of skewed parallelograms. The doors, wall region and windows are then estimated by merging the skewed parallelograms with similar color. To recognize a test image, each facet is described by its area, wall color histogram and a list of scale invariant feature transform (SIFT) descriptors. We selected a small number of SIFT features adapted with visual properties of buildings to represent the facet. To analyze multiple buildings, maximum numbers of dominant vanishing points are calculated for vertical and horizontal directions are one and five, respectively. In the first experiment, a set of 880 images is classified into building and non-building images. The second experiment is for recognizing a set of 80 test images from 500 image database. All images were taken from more than 100 buildings in Ulsan metropolitan city in South Korea under different conditions like viewpoints, camera systems, weather and seasons. We obtain 97% and 97.5% rate of correct segmentation and recognition, respectively.