A Multi-Directional Search technique for image annotation propagation

Image annotation has attracted lots of attention due to its importance in image understanding and search areas. In this paper, we propose a novel Multi-Directional Search framework for semi-automatic annotation propagation. In this system, the user interacts with the system to provide example images and the corresponding annotations during the annotation propagation process. In each iteration, the example images are clustered and the corresponding annotations are propagated separately to each cluster: images in the local neighborhood are annotated. Furthermore, some of those images are returned to the user for further annotation. As the user marks more images, the annotation process goes into multiple directions in the feature space. The query movements can be treated as multiple path navigation. Each path could be further split based on the user’s input. In this manner, the system provides accurate annotation assistance to the user - images with the same semantic meaning but different visual characteristics can be handled effectively. From comprehensive experiments on Corel and U. of Washington image databases, the proposed technique shows accuracy and efficiency on annotating image databases.

► In this paper, we proposed a Multi-Directional Search (MDS) technique. ► MDS leverages user RF to annotate images of same semantic meanings but different visual characters. ► As the query can be hierarchically split into sub-queries, the annotation speed increases in new rounds. ► MDS also considers user’s previous query intention to better exploit the intent of user feedback. ► MDS can handle uncaptioned database with unlimited vocabularies effectively by using user’s input.