Real-time video surveillance on an embedded, programmable platform

In this work, a hardware–software co-design is proposed to effectively utilize FPGA resources for a prototype of an automated video surveillance system on a programmable platform. Time-critical steps of a foreground object detection algorithm are designed and implemented in the FPGA’s logic elements to maximize parallel processing. Other non time-critical tasks are achieved by executing a high level language program on an embedded Nios-II processor. Custom and parallel processing modules are integrated into the video processing chain by a streaming protocol that aggressively utilizes on-chip memory to increase the throughput of the system. A data forwarding technique is incorporated with an on-chip buffering scheme to reduce computations and resources in the window-based operations. Other data control interfaces are achieved by software drivers that communicate with hardware controllers using Altera’s Memory-Mapped protocol. The proposed prototype has demonstrated real-time processing capability that outperforms other implementations.