An experimental study on the universality of visual vocabularies

•We show that visual vocabularies built from different datasets are universal.•We find that visual vocabularies from different datasets are nearly identical.•We investigate the number of images needed to build a universal vocabulary.•This work helps bridge the gap between bag-of-visual-words and bag-of-words.

Bag-of-visual-words has been shown to be a powerful image representation and attained success in many computer vision and pattern recognition applications. Usually for a given classification task, researchers choose to build a specific visual vocabulary, and the problem of building a universal visual vocabulary is rarely addressed. In this paper we conduct extensive classification experiments with three features on four image datasets and show that the visual vocabularies built from different datasets can be exchanged without apparent performance loss. Furthermore, we investigate the correlation between the visual vocabularies built from different datasets and find that they are nearly identical, which explains why they are universal across classification tasks. We believe that this work reveals what is behind the universality of visual vocabularies and narrows the gap between bag-of-visual-words and bag-of-words in text domain.