Hybrid image summarization by hypergraph partition

The objective of hybrid image summarization is selecting a few visual exemplars and semantic exemplars of a large-scale image collection and organizing them to represent the collection. In this paper, we present a framework for hybrid image summarization in which social images and corresponding textual information are taken as vertices in a hypergraph and the task of image summarization is formulated as the problem of hypergraph partition. A generalized spectral clustering technique is adopted to solve the hypergraph partition problem. Besides, we design two representativeness score functions to select the visual exemplars and semantic exemplars. The main advantages of the proposed approach are two-fold: (1) the hypergraph framework takes advantage of homogeneous correlations within images and tags, respectively, as well as heterogeneous relations between them, this characteristic enhances the summarization performance; and (2) we take both visual and semantic representativeness into count to select exemplars, so that the image-tag exemplars are more representative for each cluster. The experimental comparisons to the other method are conducted on some common queries for a real internet image collection. User-based evaluation demonstrates the effectiveness of the proposed approach.