Incorporating symmetry into the Lucas–Kanade framework

We present a novel variant of the Lucas–Kanade algorithm for tracking bilaterally symmetric planar objects. We show that those warping parameters which disturb the symmetry (e.g. translation perpendicular to the axis of symmetry, rotation in the plane of the object, shear) can be found without making use of a reference template, by minimizing the dissimilarity between the warped object and its mirror image. This fact enables us to decompose the tracking task of a symmetric object undergoing a complex motion into two independent tasks, i.e. two trackers running sequentially. The first, symmetry-based, tracker finds the warping parameters which disturb the object’s symmetry, while the second, a conventional Lucas–Kanade tracker which uses a reference template, completes the tracking by updating all other parameters (e.g. scale in both axes, translation parallel to the symmetry axis). The advantage of our method is that it allows decomposing an optimization process with many variables into two independent sub-processes, each one having less degrees of freedom, and hence much more robust. In case that the motion involves only parameters which disturb the symmetry, then employing the symmetry-based tracker alone is sufficient, and in this case no a priori reference template is required. We demonstrate our method by tracking a pedestrian walking along a straight path (1D translation) and a vehicle performing a turn (affine skew symmetry). The proposed algorithm is capable to cope with any warping transformation and can be generalized for the case of objects possessing higher symmetry.