Adaptive all-season image tag ranking by saliency-driven image pre-classification

• An all-season tag ranking framework is proposed to handle images with and without distinct objects.
• A simple gray histogram descriptor is proposed to describe the saliency map for image pre-classification.
• Sparse representation based neighbor voting strategy is proposed for tag relevance ranking.
• Combining sparse representation driven MIL algorithm with visual attention model is proposed to fulfill tag saliency ranking.

Social image tag ranking has emerged as an important research topic due to its application on web image search. This paper presents an adaptive all-season tag ranking algorithm which can handle the images with and without distinct object(s) using different tag ranking strategies. Firstly, based on saliency map derived from the visual attention model, a linear SVM is trained to pre-classify an image as attentive or non-attentive category by using the gray histogram descriptor on the corresponding saliency map. Then, an image with distinct object is processed by the tag saliency ranking algorithm emphasizing distinct object, which combines image saliency map with sparse representation based multi-instance learning algorithm. On the other hand, an image without distinct object can be processed by the tag relevance ranking algorithm via the sparse representation based neighbor-voting strategy. Such adaptive all-season tag ranking strategy can be regarded as taking full advantage of existing tag ranking paradigms. Experiments conducted on well-known image data sets demonstrate the effectiveness of the proposed framework.