Automatic video activity detection using compressed domain motion trajectories for H.264 videos

Most automatic event detection methods for video surveillance detect target events based on features extracted in the pixel domain. However, in practice, surveillance videos are often compressed. It is desirable to perform automatic event detection in the compressed domain directly so that the video does not need to be decoded for analysis purpose. In this paper, we investigate the use of motion trajectories for video activity detection in the compressed domain. We show it is possible to extract reliable motion trajectories directly from compressed H.264 video streams. To overcome the problems caused by unreliable motion vectors, we propose to include the information from the compressed domain prediction residuals to make the tracking more robust. We use a real world application of detecting vacant or occupied parking spaces to demonstrate the effectiveness of our proposed approach. We also demonstrate the robustness of our approach to different encoder settings, and lighting conditions.

► In this study we investigate the possibility of activity detection using compressed domain information. ► We use vacant parking space detection as an example application, where we track moving objects, classify objects into cars and pedestrian, and identify vacant parking spaces based on the trajectories all through the use of compressed domain information. ► Our experiments show that the results are mostly invariant to a variety of video encoder/decoder parameters.