Spatial–temporal convolutional neural networks for anomaly detection and localization in crowded scenes

•A mechanism for localizing dynamic regions in crowded scenes is proposed.•A spatial–temporal Convolutional Neural Network is designed to automatically extract spatial–temporal features of the crowd.•The performance of anomaly detection is improved when the analysis is concentrated on the dynamic regions only.•The anomaly events that take place in small regions are effectively detected and localized by the spatial–temporal Convolutional Neural Network.

Abnormal behavior detection in crowded scenes is extremely challenging in the field of computer vision due to severe inter-object occlusions, varying crowd densities and the complex mechanics of a human crowd. We propose a method for detecting and locating anomalous activities in video sequences of crowded scenes. The key novelty of our method is the coupling of anomaly detection with a spatial–temporal Convolutional Neural Networks (CNN), which to the best of our knowledge has not been previously done. This architecture allows us to capture features from both spatial and temporal dimensions by performing spatial–temporal convolutions, thereby, both the appearance and motion information encoded in continuous frames are extracted. The spatial–temporal convolutions are only performed within spatial–temporal volumes of moving pixels to ensure robustness to local noise, and increase detection accuracy. We experimentally evaluate our model on benchmark datasets containing various situations with human crowds, and the results demonstrate that the proposed approach surpass state-of-the-art methods.