Context-aware Discriminative Vocabulary Tree Learning for mobile landmark recognition

• Visual content and mobile device context are integrated for image recognition.
• A server-client prototype for mobile image recognition is developed.
• A Context-aware Discriminative Vocabulary Tree Learning algorithm is proposed.

Recently, mobile landmark recognition has become one of the emerging applications in mobile media, offering landmark information and e-commerce opportunities to both mobile users and business owners. Existing mobile landmark recognition techniques mainly use GPS (Global Positioning System) location information to obtain a shortlist of database landmark images nearby the query image, followed by visual content analysis within the shortlist. This is insufficient since (i) GPS data often has large errors in dense build-up areas, and (ii) direction data that can be acquired from mobile devices is underutilized to further improve recognition. In this paper, we propose to integrate content and context in an effective and efficient vocabulary tree framework. Specifically, visual content and two types of mobile context: location and direction, can be integrated by the proposed Context-aware Discriminative Vocabulary Tree Learning (CDVTL) algorithm. The experimental results show that the proposed mobile landmark recognition method outperforms the state-of-the-art methods by about 6%6%, 21%21% and 13%13% on NTU Landmark-50, PKU Landmark-198 and the large-scale San Francisco landmark dataset, respectively.