Action recognition using linear dynamic systems

In this paper, we propose a novel approach based on Linear Dynamic Systems (LDSs) for action recognition. Our main contributions are two-fold. First, we introduce LDSs to action recognition. LDSs describe the dynamic texture which exhibits certain stationarity properties in time. They are adopted to model the spatiotemporal patches which are extracted from the video sequence, because the spatiotemporal patch is more analogous to a linear time invariant system than the video sequence. Notably, LDSs do not live in the Euclidean space. So we adopt the kernel principal angle to measure the similarity between LDSs, and then the multiclass spectral clustering is used to generate the codebook for the bag of features representation. Second, we propose a supervised codebook pruning method to preserve the discriminative visual words and suppress the noise in each action class. The visual words which maximize the inter-class distance and minimize the intra-class distance are selected for classification. Our approach yields the state-of-the-art performance on three benchmark datasets. Especially, the experiments on the challenging UCF Sports and Feature Films datasets demonstrate the effectiveness of the proposed approach in realistic complex scenarios.

► We introduce the Linear Dynamic System (LDS) to action recognition.
► LDS exhibits the stationarity properties in time, so it fits for action representation.
► We propose a supervised codebook pruning method to preserve discriminative words.
► Our method is evaluated on one benchmark dataset and two challenging datasets.