Foreground detection for moving cameras with stochastic approximation

• A non-panoramic method to detect foreground objects for moving cameras is proposed.
• Probabilistic background models with full covariance matrices are employed.
• Camera movement tracking and foreground detection are done by two distinct pixel models.
• A procedure to interpolate the full covariance matrices of the pixel models is proposed.

Most foreground detection algorithms do not perform well with pan-tilt-zoom (PTZ) cameras for video surveillance and static cameras that experience vibration, since they rely on the assumption that the background does not move. Here a novel approach based on stochastic approximation learning of probabilistic mixtures is proposed. It assumes that the camera can zoom and move in both horizontal and vertical planes, and it is also adequate for egomotion sequences without abrupt changes. In other words it is a non panoramic model for moving cameras, where the camera movement allows to reuse enough background information from the previous frame. Two pixel models are used, one to follow the camera movement and the other to detect foreground objects. A procedure is developed to transform and interpolate the covariance matrices of the Gaussian mixture components as the camera moves and zooms. Moreover, a background extrapolation method is presented in order to generate new mixture models for previously unseen regions. The proposal is compared with some state-of-the-art alternatives, with competitive results both quantitatively and qualitatively.