A comparative study of object-level spatial context techniques for semantic image analysis

In this paper, three approaches to utilizing object-level spatial contextual information for semantic image analysis are presented and comparatively evaluated. Contextual information is in the form of fuzzy directional relations between image regions. All techniques, namely a Genetic Algorithm (GA), a Binary Integer Programming (BIP) and an Energy-Based Model (EBM), are applied in order to estimate an optimal semantic image interpretation, after an initial set of region classification results is computed using solely visual features. Aim of this paper is the in-depth investigation of the advantages of each technique and the gain of a better insight on the use of spatial context. For this purpose, an appropriate evaluation framework, which includes several different combinations of low-level features and classification algorithms, has been developed. Extensive experiments on six datasets of varying problem complexity have been conducted for investigating the influence of typical factors (such as the utilized visual features, the employed classifier, and the number of supported concepts) on the performance of each spatial context technique, while a detailed analysis of the obtained results is also given.

► Spatial context is efficient in improving the visual-based classification results.
► The highest performance is achieved when complex spatial constraints are used.
► Fuzzy spatial constraints lead to higher performance than binary ones.
► The spatial context performance increases when the number of objects decreases.
► The highest visual-based performance leads to the highest spatial context performance.