Cooperative object search and segmentation in Internet images

•A combined approach for cooperative object search and segmentation is proposed.•Object search helps to improve the performance of object segmentation.•Precise object segmentation improves precision rate for object search.•Structural constraint significantly improves segmentation accuracy.

We propose a combined approach for object search and segmentation in realistic Internet image collections. According to a query object, our goal is to locate and segment out those objects of interest. Our approach mainly includes two modules: the hierarchical discriminative region matching method and the iterative object segmentation algorithm. The hierarchical matching method is proposed to perform a hierarchical search to localize the seed-regions for segmentation. Then the iterative segmentation algorithm searches the optimal solution for the final segmentation, with the constraints from structural properties and seed-regions. These two modules work cooperatively because the seed-regions serve as constraints for segmentation and are also verified by segmentation results. Unlike existing search and segmentation approaches, our method produces accurate segmentation results and ignores noise images (images not containing the object of interest). The experimental results validate the advantages of our method on several benchmark datasets.