Joints kinetic and relational features for action recognition

•We propose a set of joint kinetic and relational features for action recognition.•Our features depict the characteristic of single joint, joint pair and triplet.•We conduct comprehensive experiments on three benchmarks datasets.•Our features achieve the best performance on all datasets.

High-level pose features (HLPF) have been shown to be very effective and efficient for action recognition. However, motion information has not been sufficiently mined in HLPF. In addition, the position relations of joints are limited with respect to orientation, distance and angle in HLPF. To tackle these problems, we propose a set of comprehensive features, termed joints kinetic and relational features (JKRF), for action recognition. Specifically, for each single joint, we propose a group of kinetic features to describe its velocity, speed, acceleration, acceleration rate, angular velocity, angular acceleration, kinetic energy, potential energy and total energy. For each joint pair, we propose a set of corresponding features to describe the correlation relations of velocity, acceleration, angular velocity, angular acceleration and energy change between joints. Additionally, we propose a set of corresponding features to encode distance relations in the horizontal, vertical, orientation cosine, orientation sine, eigenvector and link path directions. For each joint triplet, we also present a joint vector inner product feature, a joint vector cosine similarity feature and an area perimeter rate feature to describe their geometrical relations. We evaluate our JKRF using three datasets, and the experimental results show that JKRF consistently outperform state-of-the-art action recognition methods.