Learning discriminative trajectorylet detector sets for accurate skeleton-based action recognition

Devising a representation suitable for characterizing human actions on the basis of a sequence of pose estimates generated by an RGBD sensor remains a research challenge. We here provide two insights into this challenge. First, we show that discriminate sequence of poses typically occur over a short time window, and thus we propose a simple-but-effective local descriptor called a trajectorylet to capture the static and kinematic information within this interval. Second, we show that state of the art recognition results can be achieved by encoding each trajectorylet using a discriminative trajectorylet detector set which is selected from a large number of candidate detectors trained through exemplar-SVMs. The action-level representation is obtained by pooling trajectorylet encodings. Evaluating on standard datasets acquired from the Kinect sensor, it is demonstrated that our method obtains superior results over existing approaches under various experimental setups.