Building discriminative CNN image representations for object retrieval using the replicator equation

We present a generic unsupervised method to increase the discriminative power of image vectors obtained from a broad family of deep neural networks for object retrieval. This goal is accomplished by simultaneously selecting and weighting informative deep convolutional features using the replicator equation, commonly used to capture the essence of selection in evolutionary game theory. The proposed method includes three major steps: First, efficiently detecting features within Regions of Interest (ROIs) using a simple algorithm, as well as trivially collecting a subset of background features. Second, assigning unassigned features by optimizing a standard quadratic problem using the replicator equation. Finally, using the replicator equation again in order to partially address the issue of feature burstiness. We provide theoretical time complexity analysis to show that our method is efficient. Experimental results on several common object retrieval benchmarks using both pre-trained and fine-tuned deep networks show that our method compares favorably to the state-of-the-art. We also publish an easy-to-use Matlab implementation of the proposed method for reproducing our results.