Semi-supervised learning for refining image annotation based on random walk model

• A unified framework for refining annotation by fusing GMM with random walk.
• A semi-supervised learning is used to enhance the quality of training data.
• Gaussian mixture model is learned to estimate initial annotations.
• Integrating label and visual similarities of images associated with the labels.
• Random walk is implemented on graph to further mine the semantic correlation.

Automatic image annotation has been an active research topic in recent years due to its potential impact on both image understanding and semantic based image retrieval. In this paper, we present a novel two-stage refining image annotation scheme based on Gaussian mixture model (GMM) and random walk method. To begin with, GMM is applied to estimate the posterior probabilities of each annotation keyword for the image, during which a semi-supervised learning, i.e. transductive support vector machine (TSVM), is employed to enhance the quality of training data. Next, a label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity of images associated with the corresponding labels. In this way, it can seamlessly integrate the information from image low-level visual features and high-level semantic concepts. Followed by a random walk process over the constructed label graph is implemented to further mine the correlation of the candidate annotations so as to capture the refining results, which plays a crucial role in semantic based image retrieval. Finally, extensive experiments carried out on two publicly available image datasets bear out that this approach can achieve marked improvement in annotation performance over several state-of-the-art methods.