Unsupervised approximate-semantic vocabulary learning for human action and video classification

• Human action and video classification based on bag of visual words model.
• Parameter-less and unsupervised contextual spectral embedding framework.
• Exploiting the inter-video/image context of visual words.
• A wide variety of applications for constrained and unconstrained environments.

The paper presents a novel unsupervised contextual spectral (CSE) framework for human action and video classification. Similar to textual words, the visual word (a mid-level semantic) representation of an image or video contains a combination of synonymous words which give rise to the ambiguity of the representation. To narrow the semantic gap between visual words (mid-level semantic representation) and high-level semantics, we propose a high level representation called approximate-semantic descriptor. The experimental results show that the proposed approach for visual words disambiguation could improve the subsequent classification performance. In the paper, the approximate-semantic descriptor learning is formulated as a spectral clustering problem, such that semantically associated visual words are placed closely in low-dimensional semantic space and then clustered into one approximate-semantic descriptor. Specifically, the high level representation of human action videos is learnt by capturing the inter-video context of mid-level semantics via a non-parametric correlation measure. Experiments on four standard datasets demonstrate that our approach can achieve significantly improved results with respect to the state of the art, particularly for unconstrained environments.

Figure optionsDownload high-quality image (78 K)Download as PowerPoint slide