View invariant action recognition using projective depth

•We propose the use of projective depth for view-invariant action recognition.•Body points are decomposed into a set of projective depths.•Similarity of two actions is measured by the motion of projective depths.•Projective depths are shown to be invariant to camera parameters.•We explore different ways of extracting planes used to calculate projective depth.

In this paper, we investigate the concept of projective depth, demonstrate its application and significance in view-invariant action recognition. We show that projective depths are invariant to camera internal parameters and orientation, and hence can be used to identify similar motion of body-points from varying viewpoints. By representing the human body as a set of points, we decompose a body posture into a set of projective depths. The similarity between two actions is, therefore, measured by the motion of projective depths. We exhaustively investigate the different ways of extracting planes, which can be used to estimate the projective depths for use in action recognition including (i) ground plane, (ii) body-point triplets, (iii) planes in time, and (iv) planes extracted from mirror symmetry. We analyze these different techniques and analyze their efficacy in view-invariant action recognition. Experiments are performed on three categories of data including the CMU MoCap dataset, Kinect dataset, and IXMAS dataset. Results evaluated over semi-synthetic video data and real data confirm that our method can recognize actions, even when they have dynamic timeline maps, and the viewpoints and camera parameters are unknown and totally different.