SIFT Matching with CNN Evidences for Particular Object Retrieval

Many object instance retrieval systems are typically based on matching of local features, such as SIFT. However, these local descriptors serve as low-level clues, which are not sufficiently distinctive to prevent false matches. Recently, deep convolutional neural networks (CNN) have shown their promise as a semantic-aware representation for many computer vision tasks. In this paper, we propose a novel approach to employ CNN evidences to improve the SIFT matching accuracy, which plays a critical role in improving the object retrieval performance. To weaken the interference of noise, we extract compact CNN representations from a number of generic object regions. Then a query-adaptive method is proposed to choose appropriate CNN evidence to verify each pre-matched SIFT pair. Two different visual matching verification functions are introduced and evaluated. Moreover, we investigate the suitability of fine-tuning the CNN for our proposed approach. Extensive experiments on benchmark datasets demonstrate the effectiveness of our method for particular object retrieval. Our results compare favorably to the state-of-the-art methods with acceptable memory usage and query time.