Leveraging similarities and structure for dense representations combination in image retrieval

• An efficient and robust graph fusion framework for CBIR is proposed.
• Neighborhood structure and similarity metrics are combined in a graph.
• An efficient query adaptive similarity measure is proposed.
• Only a fraction of the memory is needed, compared to similarly performing methods.

This paper addresses the problem of content-based image retrieval in a large-scale setting. Recently several graph-based image retrieval systems have been proposed to fuse different representations, with excellent results. However, most of them use one very precise representation, which does not scale as well as global dense representations with an increasing number of images, hurting time and memory requirements as the database grows. We researched how to attain a comparable precision, while greatly reducing the memory and time requirements by avoiding the use of a main precise representation. To accomplish this objective, we proposed a novel graph-based query fusion approach—where we combined several compact representations based on aggregating local descriptors such as Fisher Vectors—using distance and neighborhood information jointly to evaluate the individual importance of each element in a query adaptive manner. The performance was analyzed in different time and memory constrained scenarios, ranging from less than a second to several seconds for the complete search process while needing only a fraction of the memory compared to other similar performing methods. Experiments were performed on 4 public datasets, namely UKBench, Holidays, Corel-5K and MIRFLICKR-1M, obtaining state-of-the-art effectiveness.