Intelligent video surveillance beyond robust background modeling

•Extremely complex environments: sudden illumination changes are tackled.•Evaluation metrics for perimeter protection solutions are analyzed.•Constraints to classify false positives are learnt from example; no hand-crafted rules.•Global features are extracted to make machines learn complex scenes.•Experiments to verify our proposal have been conducted.

The increasing number of video surveillance cameras is challenging video control systems. Monitoring centers require tools to guide the process of supervision. Different video analysis methods have effectively met the main requirements from the industry of perimeter protection. High accuracy detection systems are able to process real time video on affordable hardware. However some problematic environments cause a massive number of false alerts. Many approaches in the literature do not consider this kind of environments while others use metrics that dilute their impact on results. An intelligent video solution for perimeter protection must select and show the cameras which are more likely witnessing a relevant event but systems based only on background modeling tend to give importance to problematic situations no matter if an intrusion is taking place or not. We propose to add a module based on machine learning and global features, bringing adaptability to the video surveillance solution, so that problematic situations can be recognized and given the right priority. Tests with thousands of hours of video show how good an intruder detector can perform but also how a simple fault in a camera can flood a monitoring center with alerts. The new proposal is able to learn and recognize events such that alerts from problematic environments can be properly handled.