Fixed-point Gaussian Mixture Model for analysis-friendly surveillance video coding

• The fGMM is mathematically derived to eliminate the floating-point calculations.
• A division simulation algorithm is proposed to replace the division operations.
• The fGMM is implemented in FPGA to process HD videos in real-time.

With the recent explosion in the use of video surveillance in security, social and industrial applications, it is highly desired to develop “smart” cameras which are capable of not only supporting high-efficiency surveillance video coding but also facilitating some content analysis tasks such as moving object detection. Usually, background modeling is one of fundamental pre-processing steps in many surveillance video coding and analysis tasks. Among various background models, Gaussian Mixture Model (GMM) is considered as one of the best parametric modeling methods for both video coding and analysis tasks. However, a number of floating-point calculations and division operations largely limit its application in the hardware implementation (e.g., FPGA, SOC). To address this problem, this paper proposes a fixed-point Gaussian Mixture Model (fGMM), which can be used in the hardware implementation of the analysis-friendly surveillance video codec in smart cameras. In this paper, we first mathematically derive a fixed-point formulation of GMMs by introducing several integer variables to replace the corresponding float ones in GMM so as to eliminate the floating-point calculations, and then present a division simulation algorithm and an approximate calculation to replace the division operations. Extensive experiments on the PKU-SVD-A dataset show that fGMM can achieve comparable performance with the float GMM on both surveillance video coding and object detection tasks, and outperforms several state-of-the-art methods remarkably. We also implemented fGMM in FPGA. The result shows that the FPGA implementation of our fGMM can process HD videos in real-time, just requiring 140 MHz user logic and 622 MHz DDR3 memory with 64-bit data bus.