MLRank: Multi-correlation Learning to Rank for image annotation

• We formulate image annotation into a Multi-correlation Learning to Rank framework.
• It is a semi-supervised learning method leveraging labeled and unlabeled images.
• The ranking function is estimated by leveraging labeled and unlabeled images.
• The ranking function explores correlation consistencies between images and tags.
• Experiments on three benchmarks demonstrate the superior performance of our work.

In this paper, we formulate image annotation as a Multi-correlation Learning to Rank (MLRank) problem, i.e., ranking the relevance of tags to an image considering the visual similarity and the semantic relevance. Unlike typical learning to rank algorithms, which assume that the ranking objects are independent, we attempt to rank relational data by exploring the consistency between “visual similarity” and “semantic relevance”. The consistency means that similar images are usually annotated with relevant tags to reflect similar semantic themes, and vice versa. We define the two cases as the image-bias consistency and the tag-bias consistency respectively, which are both formulated into the optimization problem for rank learning. To obtain an explicit solution of the ranking model, we relax the optimization problem in two manners by attaching the constraints corresponding to the image-bias and tag-bias consistency with different sequential orders respectively, which lead to a uniform ranking model. Experimental results show that the proposed MLRank method outperforms the state-of-the-arts on three benchmarks including Corel5K, IAPR TC12 and NUS-WIDE.