Discriminative human action classification using locality-constrained linear coding

•We propose using the locality-constrained linear coding for action classification.•Our sequence descriptor includes cell, block, and subsequence descriptors.•We use maximum pooling and a logistic regression classifier to encode each sequence.•We demonstrate the effectiveness of our algorithm on both depth and RGB videos.

We propose a Locality-constrained Linear Coding (LLC) based algorithm that captures discriminative information of human actions in spatio-temporal subsequences of videos. The input video is divided into equally spaced overlapping spatio-temporal subsequences. Each subsequence is further divided into blocks and then cells. The spatio-temporal information in each cell is represented by a Histogram of Oriented 3D Gradients (HOG3D). LLC is then used to encode each block. We show that LLC gives more stable and repetitive codes compared to the standard Sparse Coding. The final representation of a video sequence is obtained using logistic regression with ℓ2 regularization and classification is performed by a linear SVM. The proposed algorithm is applicable to conventional and depth videos. Experimental comparison with ten state-of-the-art methods on three depth video and two conventional video databases shows that the proposed method consistently achieves the best performance.