Unified optimization framework for localization and tracking of multiple targets with multiple cameras

We present a method for three-dimensional (3D) localization and tracking of multiple targets by using images from multiple cameras with overlapping views. Most recent methods make an assumption on a flat ground plane and determine the 3D grounding location of each target based on this assumption. In contrast, we aim to find 3D locations of multiple head trajectories, regardless of their standing or sitting status without the flat ground plane assumption. For this purpose, we suggest a unified optimization formulation, which solves two coupled problems simultaneously: the spatio-temporal data association problem and the 3D trajectory estimation problem. To handle a large solution space, we develop an efficient optimization scheme that alternates the solving procedures between two coupled problems with a reasonable computational load. In the unified optimization formulation, we design a new cost function that describes 3D physical properties of each target. The experiments illustrate that the proposed method outperforms the state-of-the-art methods in 3D localization and tracking performance.