System-on-a-chip (SoC)-based hardware acceleration for foreground and background identification

The rapid growth of embedded vision applications and accessibility in recent years has instigated a philosophical shift in algorithm and implementation design for artificial intelligence. With the popularization of high-definition video, the amount of data available to be processed has also increased substantially, posing massive computational and communication demands. Hardware acceleration through specialization has received renewed interest in recent years; such acceleration has generally been implemented using two chips, with the image signal processing (ISP) part being performed by a DSP, a GPU or an FPGA and the video content analytics (VCA) part being executed by a processor. GPUs consume a substantial amount of power; thus, it is challenging to deploy them in embedded environments. However, the new generation of SoC-FPGAs that are fabricated with both the microprocessor and FPGA on a single chip consumes less power and can be built into small systems, thereby offering an attractive platform for embedded applications. This study presents the hardware acceleration of a real-time adaptive background and foreground identification algorithm in a SoC, including the capture, processing and display stages. The algorithm can be performed in either 2D or 3D space. The proposed platform uses photometric invariant color, depth data and local binary patterns (LBPs) to distinguish background from foreground. The system uses minimal cell resources, an elastically pipelined architecture is used to absorb variations in processing time, and each pipeline stage is optimized to use the available FPGA primitives. Additionally, the communication-centric architecture used in this work simplifies the implementation of embedded vision algorithms.