A new framework of action recognition with discriminative parts, spatio-temporal and causal interaction descriptors

To improve action recognition performance, a novel discriminative spectral clustering method is firstly proposed, by which the candidate parts with the internal trajectories being close in spatial position, consistent in appearance and similar in motion velocity are mined. Furthermore, the discriminative constraint is introduced to select discriminative parts. Meanwhile, by fully considering the local and global distributions of data, a new similarity matrix is constructed, which enhances clustering effect. Secondly, the spatio-temporal interaction descriptor and causal interaction descriptor are constructed respectively, which fully mine the spatio-temporal and implicit causal interactive relationships between parts. Finally, a new framework is proposed. By associating the discriminative parts, spatio-temporal and causal interaction descriptors together as the inputs of Latent Support Vector Machine (LSVM), the correlations between action categories and action parts as well as interaction descriptors are mined. Consequently, accuracy is enhanced. The extensive and adequate experiments demonstrate the effectiveness of the proposed method.