Weighted feature trajectories and concatenated bag-of-features for action recognition

Key-point trajectory based approaches to recognizing human actions in realistic videos have recently shown promising results. However, their coverage of the entire actor is not sufficient for describing human actions, and the trajectories often disappear due to partial occlusions. In this paper, we propose a novel approach based on weighted feature trajectories and concatenated bag-of-features (BOF) using weighted spatio-temporal descriptors to overcome these limitations. To capture spatio-temporal regions of interest (ROI), we first build scale-invariant feature transform (SIFT) trajectories based on matching SIFT descriptors. Then, we extract particle trajectories around the SIFT trajectories using dense optical flow to retain ROI subsidiarily and to give a sound coverage of the actors. For balancing the leverages between SIFT and particle trajectories, a weighting scheme is presented according to the accumulated global distribution of feature points. Furthermore, we propose the concatenated two distinct SIFT and particle BOFs with weighted spatio-temporal descriptors for action recognition. Experimental results demonstrate the effectiveness of this approach in challenging real-world action datasets such as KTH, UCF sports, and TV human interaction.