Context-aware vocabulary tree for mobile landmark recognition

•Vocabulary tree is fast and scalable for mobile landmark recognition.•Context information can help to learn a more discriminative codebook.•Information gain based method is effective for codeword selection.•Inverted file structure can speed up the image matching process.

This paper presents an effective approach that incorporates contextual information into vocabulary tree learning for mobile landmark recognition. For most existing mobile landmark recognition works, the context information (GPS or direction) is mainly used to reduce the search space in a heuristic and insufficient manner. Some recent work uses the context information for codebook learning but only the GPS information is explored. We propose an effective mobile landmark recognition approach which exploits both context (direction and location) and content information for vocabulary tree learning and image recognition. The proposed approach has two major contributions: (i) it proposes an information gain-based codeword discrimination learning method to evaluate the discriminative capability of each direction-aware codeword, as generated by a context-aware vocabulary tree, and (ii) it develops a context-aware image scoring technique based on an inverted file structure that speeds up the image matching process greatly. Experimental results on the NTU and San Francisco database show that the proposed method can achieve good recognition performance with fast speed.