Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management

In this paper we present an effective and flexible real-time video analysis system aiming at a wide range of outdoor surveillance and monitoring scenarios, in which robust detection and tracking objects of interest (pedestrians and/or vehicles) are essential to detecting events and understanding scene changes. The approaches employed address properly the challenges found in typical outdoor scenes such as localised and global lighting changes, variations in object size and views, occlusions, and complex object motion and so on. The contributions include use of an effective two-layer shadow removal scheme based on brightness distortion in different colour channels, blob-based object tracking and occlusion handling via multi-state transition management. The system works equally well on highly compressed (JPEG + MPEG-4) video streams as well as raw video data; in the former case, it achieves about 18 frames per second with software MPEG-4 decoder in a modern PC. And the real-time constraint renders most complex tracking algorithms less attractive.