Automatic 3D tracking system for large swarm of moving objects

• We present a 3D tracking for a large number of targets.
• We propose a framework that resolves detection and stereo ambiguities effectively.
• We propose a 3D tracker that tracks targets in 3D space from 2D cues.
• We have conducted systematical evaluation experiments.

Natural systems such as bird flocks, fish schools and insect swarms consist of a large group of moving individuals. For many years, scientists have been interested in the complex 3D motion patterns and dynamics they exhibit, trying to discover enlightening rules and causes behind them. Unfortunately, the lack of effective techniques to accurately measure the real 3D trajectories of the individuals had limited the quantitative study on these systems. We propose in this paper an automatic tracking system which is able to track a large number of tiny animals in a 3D volume with multiple cameras. Most visual details of such targets are lost in the captured images because of limited image resolution, and the remainder can be easily corrupted due to frequent occlusion or motion blur, which makes it difficult to establish cross-view and cross-frame correspondences. We formulate the problem as a repeated process of hypothesis generation and verification. Hypotheses are generated when cross-view matching ambiguities occur and are verified at an efficient 3D tracking stage where targets are modeled in 3D space and weak yet existing visual information from multi-view video streams are furthest collected. The whole system is fully automatic in dealing with variable number of targets and robust against detection and matching errors.