Look, listen and find: A purely audiovisual approach to online videos geotagging

• The proposed system pioneers purely audiovisual geotagging at Earth scale.
• Our proposal builds on our previous expertise on environmental sound recognition.
• It outperforms all MediaEval2011 audiovisual and visual-content based geotaggers.

Tagging videos with the geo-coordinates of the place where they were filmed (i.e. geotagging) enables indexing online multimedia repositories using geographical criteria. However, millions of non geotagged videos available online are invisible to the eyes of geo-oriented applications, which calls for the development of automatic techniques for estimating the location where a video was filmed. The most successful approaches to this problem largely rely on exploiting the textual metadata associated to the video, but it is quite common to encounter videos with no title, description nor tags. This work focuses on this latter adverse scenario and proposes a purely audiovisual approach to geotagging based on audiovisual similarity retrieval, modality fusion and cluster density. Using a subset of the MediaEval 2011 Placing task data set, we evaluate the ability of several visual and acoustic features for estimating the videos location both separately and jointly (via fusion at feature and at cluster level). The optimally configured version of the proposed system is capable of geotagging videos within 1 km of their real location at least 4 times more precisely than any of the audiovisual and visual content-based participants in the MediaEval 2011 Placing task.