Gradient-layer feature transform for action detection and recognition

•We introduce a powerful gradient based method to extract image features.•We propose an efficient projection based approach for action detection.•We create an effective correspondence fusion technique for action representation.

Exploring action feature representation in consecutive video frames is a basic but critical issue in the area of computer vision. This paper presents a principled technique transforming gradient-based features into coherent spatial-temporal descriptors for action detection and recognition. Specifically, Gaussian convolution based technique is first applied to extract spatial features of each image frame on gradient layer, based on which the spatial features are further processed according to the forward-backward frame difference and correspondence fusion between frames for frame sequence representation. Furthermore, region of actions is labeled via thresholding the projection of difference features in horizontal-vertical direction while action types are classified via learning the fused features. We evaluate our approach on samples from KTH, Weizmann, UCF Sports dataset and ChangeDetection.NET dataset 2014, which demonstrates its applicability and effectiveness.