Multi-level feature representations for video semantic concept detection

Video semantic concept detection is a fundamental problem with many practical applications such as concept-based video retrieval. The major challenge of concept detection lies in the existence of the well-known semantic gap between the low-level visual features and the user׳s semantic interpretation of visual data. To bridge the semantic gap, in this paper we propose to promote low-level visual features to middle-level representations, expecting that the underlying latent semantic aspects of image data can be discovered, and such aspects can better model the semantic of images. Specifically, we employ latent Dirichlet allocation (LDA) approach to cluster the image data into semantic topics and the distributions of image low-level features on such topics are used as the middle-level feature vectors of images. Meanwhile, a recently developed more efficient probabilistic representation of low-level features, i.e., Fisher Vector is used to complement the LDA representation for video concept detection. The experimental results on the TRECVID 2013 Semantic Indexing dataset have demonstrated the effectiveness of the proposed approach.