Finding more relevance: Propagating similarity on Markov random field for object retrieval

•A Markov random field based probabilistic framework is proposed for image retrieval.•A belief propagation algorithm is proposed for the inference of Markov random field.•A new image retrieval system is designed based on the proposed framework.•Our system overcomes the failures caused by the differences of imaging conditions.•Our system achieves the state-of-the-art results on three public datasets.

To retrieve objects from large corpus with high accuracy is a challenging task. In this paper, we propose a Markov random field (MRF) based probabilistic retrieval framework. In this framework, the similarities between the query image and dataset images are modeled as the likelihood and the relationships among the images in the dataset are modeled as the prior. Then, the prior and the likelihood are combined to improve retrieval performance. Further, we present an approximate belief propagation algorithm as well as a subgraph extraction algorithm for efficient inference in MRF. Finally, we design a new image retrieval system under our framework. This system can be considered as an extended bag-of-visual-words retrieval system with the probabilistic based re-ranking module. We evaluate our method on three standard datasets: Oxford-5K, Oxford-105K and Paris-6K. The experimental results show that the proposed system significantly improves the retrieval accuracy on these datasets and exceeds the state-of-the-art results.