Visual query expansion with or without geometry: Refining local descriptors by feature aggregation

• Novel query expansion method aligned with Hamming Embedding which individually matches features.
• Refinement of local description instead of global one in the previous query expansion methods.
• Descriptor aggregation providing more compact and more robust local representation.
• No need for spatial matching to identify good candidates for query expansion.

This paper proposes a query expansion technique for image search that is faster and more precise than the existing ones. An enriched representation of the query is obtained by exploiting the binary representation offered by the Hamming Embedding image matching approach: the initial local descriptors are refined by aggregating those of the database, while new descriptors are produced from the images that are deemed relevant.The technique has two computational advantages over other query expansion techniques. First, the size of the enriched representation is comparable to that of the initial query. Second, the technique is effective even without using any geometry, in which case searching a database comprising 105k images typically takes 79 ms on a desktop machine. Overall, our technique significantly outperforms the visual query expansion state of the art on popular benchmarks. It is also the first query expansion technique shown effective on the UKB benchmark, which has few relevant images per query.