Human action recognition with salient trajectories

•Trajectory saliency is formulated for human action recognition.•Trajectory appearance saliency is proposed with respect to static visual aspect.•Trajectory motion saliency is proposed with respect to dynamic visual aspect.•Appearance and motion saliency are combined.

Recognizing human actions in video sequences is attracting much attention, and this paper aims to deal with the problem of action recognition with salient trajectories. First, two kinds of trajectory saliency values, appearance and motion saliency, are calculated and combined to capture complementary information. Secondly, the combined saliency is utilized to prune redundant trajectories, and a compact and discriminative set of trajectories is obtained. Finally, kernel histograms are applied for the description of salient trajectories and human actions are classified by the bag-of-words approach. The proposed approach is validated on three public datasets including KTH, ADL, and UCF. Experimental results show that the method achieves superior results on the KTH and ADL datasets and comparable results with other state-of-the-art methods on the UCF dataset.