Social image tagging using graph-based reinforcement on multi-type interrelated objects

Social image tagging is becoming increasingly popular with the development of social website, where images are annotated with arbitrary keywords called tags. Most of present image tagging approaches are mainly based on the visual similarity or mapping between visual feature and tags. However, in the social media environment, images are always associated with multi-type of object information (i.e., visual content, tags, and user contact information) which makes this task more challenging. In this paper, we propose to fuse multi-type of information to tag social image. Specifically, we model social image tagging as a “ranking and reinforcement” problem, and a novel graph-based reinforcement algorithm for interrelated multi-type objects is proposed. When a user issue a tagging request for a query image, a candidate tag set is derived and a set of friends of the query user is selected. Then a graph which contains three types of objects (i.e., visual features of the query image, candidate tags, and friend users) is constructed, and each type of objects are initially ranked based on their weight and intra-relation. Finally, candidate tags are re-ranked by our graph-based reinforcement algorithm which takes into consideration both inter-relation with visual features and friend users, and the top ranked tags are saved. Experiments on real-life dataset demonstrate that our algorithm significantly outperforms state-of-the-art algorithms.

► A object relation graph to fuse different object. ► A iterated form of the reinforcement model. ► Reinforcement of tags based on both visual feature and social contact information.