Particle filters and occlusion handling for rigid 2D–3D pose tracking

• A robust algorithm for 2D visual tracking and 3D pose estimation is proposed.
• We focus on partial occlusions that distort the region properties of an object.
• Optimal pose of an object is estimated via particle filters in a decoupled manner.
• The degree of trust between system’s predictions and measurements is controlled.
• The resulting methodology improves tracking performance in clutter and occlusion.

In this paper, we address the problem of 2D–3D pose estimation. Specifically, we propose an approach to jointly track a rigid object in a 2D image sequence and to estimate its pose (position and orientation) in 3D space. We revisit a joint 2D segmentation/3D pose estimation technique, and then extend the framework by incorporating a particle filter to robustly track the object in a challenging environment, and by developing an occlusion detection and handling scheme to continuously track the object in the presence of occlusions. In particular, we focus on partial occlusions that prevent the tracker from extracting an exact region properties of the object, which plays a pivotal role for region-based tracking methods in maintaining the track. To this end, a dynamical choice of how to invoke the objective functional is performed online based on the degree of dependencies between predictions and measurements of the system in accordance with the degree of occlusion and the variation of the object’s pose. This scheme provides the robustness to deal with occlusions of an obstacle with different statistical properties from that of the object of interest. Experimental results demonstrate the practical applicability and robustness of the proposed method in several challenging scenarios.