Hierarchical Bayesian models for unsupervised scene understanding

• Two novel hierarchical models for unsupervised scene understanding are presented.
• We investigate hierarchical structures for modelling context within our models.
• These models are compared against unsupervised, weakly and fully supervised models.
• Our new models are competitive with the supervised models for scene recognition.
• We also show the models operating on a large underwater dataset collected by a robot.

For very large datasets with more than a few classes, producing ground-truth data can represent a substantial, and potentially expensive, human effort. This is particularly evident when the datasets have been collected for a particular purpose, e.g. scientific inquiry, or by autonomous agents in novel and inaccessible environments. In these situations there is scope for the use of unsupervised approaches that can model collections of images and automatically summarise their content. To this end, we present novel hierarchical Bayesian models for image clustering, image segment clustering, and unsupervised scene understanding. The purpose of this investigation is to highlight and compare hierarchical structures for modelling context within images based on visual data alone. We also compare the unsupervised models with state-of-the-art supervised and weakly supervised models for image understanding. We show that some of the unsupervised models are competitive with the supervised and weakly supervised models on standard datasets. Finally, we demonstrate these unsupervised models working on a large dataset containing more than one hundred thousand images of the sea floor collected by a robot.