Human activity recognition by separating style and content

•Addresses problem of recognizing human activities and underling execution styles (people).•Combines asymmetric bilinear modeling with conventional action recognition.•Uses only motion cues.•Achieves high recognition accuracy for Kinect depth data, mocap, and motion history volume.

Studies in psychophysics suggest that people tend to perform different actions in their own style. This article deals with the problem of recognizing human actions and the underlying execution styles (actors) in videos. We present a hierarchical approach that is based on conventional action recognition and asymmetrical bilinear modeling. In particular, we employ bilinear factorization on the tensorial representation of the action videos to characterize styles of performing different actions. Our approach is solely based on the dynamics of the underlying activity. The model is evaluated on the IXMAS and the Berkeley-MHAD data sets using different modalities based on optical motion capture, Kinect depth videos, and 3D motion history volumes. In each case high recognition accuracy is achieved in comparison to the symmetric bilinear modeling and the Nearest Neighbor classification.