Informative joints based human action recognition using skeleton contexts

•Our informative joints based method eliminates noise by ignoring the joints of small contributions.•We use binned pairwise space distribution of informative joints to build discriminative skeleton contexts.•Our representation is strongly invariant to individual size and shape, and viewpoint.•Improved affinity propagation was used to automatically find the exemplar features without worrying about bad initialization.•The proposed approach is discriminative for similar human action recognition and well adapted to the intra-class variation.

The launching of Microsoft Kinect with skeleton tracking technique opens up new potentials for skeleton based human action recognition. However, the 3D human skeletons, generated via skeleton tracking from the depth map sequences, are generally very noisy and unreliable. In this paper, we introduce a robust informative joints based human action recognition method. Inspired by the instinct of the human vision system, we analyze the mean contributions of human joints for each action class via differential entropy of the joint locations. There is significant difference between most of the actions, and the contribution ratio is highly in accordance with common sense. We present a novel approach named skeleton context to measure similarity between postures and exploit it for action recognition. The similarity is calculated by extracting the multi-scale pairwise position distribution for each informative joint. Then feature sets are evaluated in a bag-of-words scheme using a linear CRFs. We report experimental results and validate the method on two public action dataset. Experiments results have shown that the proposed approach is discriminative for similar human action recognition and well adapted to the intra-class variation.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide