Action recognition using direction-dependent feature pairs and non-negative low rank sparse model

In this paper, we propose to use direction-dependent feature pairs (DDFP) to represent actions and a novel non-negative low rank sparse model (NLRM) is developed to encode the features. We summarize our main contributions into three aspects. First, for a video we apply eight different directions to describe the spatio-temporal relations between features, and construct directional feature pairs according to their relative positions. Second, we present a non-negative low rank sparse model which incorporates the low rank term and the non-negative constraint. Our model can not only ensure the consistency of similar DDFP by the low rank term, but also enforce the sparsity of coding coefficients by the modified l2,1l2,1-norm regularization. Third, we utilize a direction-specific dictionary for each direction and encode DDFP of a specific direction by the corresponding dictionary. A video is finally represented by the concatenation of each direction׳s pooling result. Experimental results on the KTH, Weizmann and UCF sports dataset show the effectiveness of our proposed framework for human action recognition.