A query expansion framework in image retrieval domain based on local and global analysis

We present an image retrieval framework based on automatic query expansion in a concept feature space by generalizing the vector space model of information retrieval. In this framework, images are represented by vectors of weighted concepts similar to the keyword-based representation used in text retrieval. To generate the concept vocabularies, a statistical model is built by utilizing Support Vector Machine (SVM)-based classification techniques. The images are represented as “bag of concepts” that comprise perceptually and/or semantically distinguishable color and texture patches from local image regions in a multi-dimensional feature space. To explore the correlation between the concepts and overcome the assumption of feature independence in this model, we propose query expansion techniques in the image domain from a new perspective based on both local and global analysis. For the local analysis, the correlations between the concepts based on the co-occurrence pattern, and the metrical constraints based on the neighborhood proximity between the concepts in encoded images, are analyzed by considering local feedback information. We also analyze the concept similarities in the collection as a whole in the form of a similarity thesaurus and propose an efficient query expansion based on the global analysis. The experimental results on a photographic collection of natural scenes and a biomedical database of different imaging modalities demonstrate the effectiveness of the proposed framework in terms of precision and recall.

Research highlights
► Investigate whether the query expansion approach is suitable for image domain based on the image representation as the “bag of concepts” model.
► Propose automatic query expansion approaches based on both the local and global analysis.
► Exploit the concept–concept correlations by analyzing a local clustering method.
► Taking into account the co-occurrence information and the metrical constraints based on the neighborhood proximity between the concepts.
► Analyze the concept similarities in the collection as a whole for query expansion based on the global analysis.