An energy-based model for region-labeling

This paper introduces an energy-based model (EBM) for region labeling that takes advantage of both context and semantics present in segmented images.The proposed method refines the output of multiclass classification methods that are based on the one-vs-all (OVA) formulation. Intuitively, the EBM maximizes the semantic cohesion among labels assigned to neighboring regions; that is, a tradeoff between label-association information and the predictions from the base classifier. Additionally, we study the suitability of OVA classification for the region labeling task. We report experimental results of our methods in 12 heterogeneous data sets that have been used for the evaluation of different tasks besides region labeling. On the one hand, our results reveal that the OVA approach offers an important potential of improvement in terms of labeling performance that can be exploited by refinement techniques similar to ours. On the other hand, experimental results show that our EBM improves the labeling provided by the base classifier. The EBM is highly efficient and it can be applied without modifications to different data sets. The heterogeneity of the considered databases shows the generality of our approach and its robustness to different scenarios. Our results are superior to other techniques that have been tested in the same collections. Furthermore, results on image retrieval show that the labels generated with our EBM can be helpful for annotation-based image retrieval.

Research highlights
► Region labeling is successfully faced as OVA multiclass classification.
► We study the benefits offered by OVA classification to region labeling.
► Introduce model that refines the output of a classifier by using co-occurrences.
► Our method is easier to implement and shows better performance than similar techniques.
► We show that automatic labels can give support to image retrieval.
► An extensive evaluation is performed across heterogeneous data sets.