A multiple object tracking approach that combines colour and depth information using a confidence measure

Multiple object tracking is a difficult task, especifically when there is not an explicit model of the object being tracked or when it is not possible to estimate the background of the scene. This paper proposes a novel approach for multiple target tracking. It works without background information and uses an original method that merges colour and depth information. The fusion of both pieces of information is created taking into account a confidence measure about the depth information. The method proposed employs a multiple particle filter approach in which particle weights are modified by an interaction factor in order to avoid the “coalescence” problem. In addition, the method performs as a pure colour-based technique when no disparity information is available, and takes advantage of depth information to enhance tracking whenever it is possible. Our technique is compared with two pure colour-based tracking approaches (the particle filtering method proposed by Nummiaro et al. [Nummiaro, K., Koller-Meier, E., Van Gool, L., 2003. An adaptive color-based particle filter. Image and Vision Computing, 21, 99–110] and the Kalman/mean-shift tracker [Comaniciu, D., Ramesh, V. 2000. Mean shift and optimal prediction for efficient object tracking. In: IEEE International Conference on Image Processing (ICIP’00), vol. 3, pp. 70–73]) and a pure stereo-based approach derived from our problem formulation. The performance of the four algorithms is tested using several colour-with-depth sequences of images showing different coloured targets in complex situations. The results show that our proposal is able to track the targets in case of complex backgrounds and to properly determine the size of their projections in the camera image (while the other methods fail). Besides, the proposed method is fast enough for real-time applications and the use of 3D information helps to track several targets simultaneously without confusing their identities.