Automatic image annotation using semi-supervised generative modeling

• We propose a modified EM algorithm to incorporate unlabeled images in training phase.
• Grouping images using spectral clustering improves prototypes and models of concepts.
• For noisy annotated images, semi-supervised mixture model outperforms graph learning.
• Incorporating unlabeled images will improve annotation performance significantly.

Image annotation approaches need an annotated dataset to learn a model for the relation between images and words. Unfortunately, preparing a labeled dataset is highly time consuming and expensive. In this work, we describe the development of an annotation system in semi-supervised learning framework which by incorporating unlabeled images into training phase reduces the system demand to labeled images. Our approach constructs a generative model for each semantic class in two main steps. First, based on Gamma distribution, a generative model is constructed for each semantic class using labeled images in that class. The second step incorporates the unlabeled images by using a modified EM algorithm to update parameters of the constructed generative models. Performance evaluation of the proposed method on a standard dataset reveals that using unlabeled images will result in considerable improvement in accuracy of the annotation systems when a limited number of labeled images for each semantic class are available.