Image annotation using high order statistics in non-Euclidean spaces

•We utilize the high-order dissimilarity among a triplet of images to depict image visual content.•We utilize a maximum a posteriori algorithm to achieve relevance of each tag for input new images.•We utilize the properties of images in non-Euclidean space to achieve precise annotations.

Automatic image annotation is a promising way to achieve more effective image retrieval and image analysis by using keywords associated to the image content. Due to the semantic gap between low-level visual features and high-level semantic concepts of an image, however, the performances of many existing algorithms are not so satisfactory. In this paper, a novel image classification scheme, named high order statistics based maximum a posterior (HOS-MAP), is proposed to deal with the issue of image annotation. To bridge the gap between human judgment and machine intelligence, the proposed scheme first constructs a dissimilarity representation for each image in a non-Euclidean space; then, the information of dissimilarity diffusion distribution for each image is achieved with respect to the high-order statistics of a triplet of nearest neighbor images; finally, a maximum a posteriori algorithm with the information of Gaussian Mixture Model and dissimilarity diffusion distribution is adopted to estimate the relevance between each annotation and an input un-annotated image. Experimental results on a general-purpose image database demonstrate the effectiveness and efficiency of the proposed automatic image annotation scheme.