Multi-perspective and multi-modality joint representation and recognition model for 3D action recognition

In this paper, we proposed multi-perspective and multi-modality discriminated and joint representation and recognition model for 3D action recognition. Specifically, for depth and RGB image sequence, we construct a novel difference motion history image, and then propose multi-perspective projections to capture the target motion process, after that, pyramid histogram of orientated gradients is extracted for each projection to describe the target motion, finally, multi-perspective and multi-modality discriminated and joint representation and recognition model is proposed to recognize human action. Large scale experimental results on challenging and public DHA 3D and MSR-Action3D action datasets show that the performances of our difference motion history image on two modalities are much better than traditional motion history image, at the same time, our description scheme is also very robust and efficient, what is more, our proposed multi-perspective and multi-modality discriminated and joint representation and recognition model further improves the performance, which outperforms the state-of-the-art methods, and whose best performances on MSR-Action3D and DHA datasets reach 90.5% and 98.2% respectively.