Boosting histograms of descriptor distances for scalable multiclass specific scene recognition

We present an unconventional way of using keypoints in the form of histograms of keypoint descriptor distances. Descriptor distances are often exhaustively computed between sets of keypoints, but besides finding the k-smallest distances the structure of the distribution of these distances has been largely overlooked. We highlight the potential of such information in the task of specific scene recognition. Discriminative scene signatures in the form of histograms of keypoint descriptor distances are constructed in a supervised manner. The distances are computed between properly selected reference keypoints and the keypoints detected in the input image. The signature is low dimensional, computationally cheap to obtain, and can distinguish a large number of scenes. We introduce a scheme based on Multiclass AdaBoost to select the appropriate reference keypoints. The result is a scalable multiclass specific scene classifier capable of processing a large number of scene classes at a fraction of the time required for methods based on exhaustive keypoint matching. We test the idea on 3 datasets for specific scene recognition and report the obtained results.

Research Highlights
► In image feature matching, keypoint descriptor distances are often exhaustively computed between sets of keypoints.
► However, previously only the k-smallest of such distances are of interest.
► We show that the other (usually discarded) descriptor distances also reveal useful information about the image database.
► In particular we show how to improve specific scene recognition using our novel image feature based on histogramming descriptor distances.