On the improvement of human action recognition from depth map sequences using Space–Time Occupancy Patterns

• A new visual representation for 3D action recognition from depth map sequences.
• A scheme for on-line recognition with automatic segmentation and time alignment.
• Combination of depth maps with skeletons to obtain view invariance.
• Results show improvements in comparison with state-of-the-art methods.

We present a new visual representation for 3D action recognition from sequences of depth maps. In this new representation, space and time axes are divided into multiple segments to define a 4D grid for each depth map sequences. Each cell in the grid is associated with an occupancy value which is a function of the number of space–time points falling into this cell. The occupancy values of all the cells form a high dimensional feature vector, called Space–Time Occupancy Pattern (STOP). We then perform dimensionality reduction to obtain lower-dimensional feature vectors. The advantage of STOP is that it preserves spatial and temporal contextual information between space and time cells while being flexible enough to accommodate intra-action variations. Furthermore, we combine depth maps with skeletons in order to obtain view invariance and present an automatic segmentation and time alignment method for on-line recognition of depth sequences. Our visual representation is validated with experiments on a public 3D human action dataset.