Kinematic sets for real-time robust articulated object tracking

In this article, a new approach is given for real-time visual tracking of a class of articulated non-rigid objects in 3D. The main contribution of this paper consists in symmetrically modeling the motion and velocity of an articulated object via a novel kinematic set approach. This is likened to a Lagrange–d'Alembert formulation in classical physics. The advantages of this new model over pre-existing methods include improved precision, robustness and efficiency, leading to real-time performance. Furthermore, a general class of mechanical joints can be considered and the method can track objects where previous approaches have failed due to a lack of visual information. In summary, a joint configuration is modeled by using Pfaffian velocity constraints. The configuration and location of a joint is then used to build a general Jacobian Matrix, which relates individual rigid body velocities (twists) to an underlying minimal subspace. A closed loop control law is then derived in order to minimize a set of distance errors in the image and estimate the system parameters. The tracking is locally based upon efficient distance criterion. Experimental results show prismatic, rotational and helical type links and eight general parameters. A statistical M-estimation technique is applied to improve robustness. A monocular camera system was used as a real-time sensor to verify the theory.