Generalizing the Lucas–Kanade algorithm for histogram-based tracking

We present a novel histogram-based tracking algorithm, which is a generalization of the template matching Lucas–Kanade algorithm (and in particular of the inverse compositional version which is more efficient). The algorithm does not make use of any spatial kernel. Instead, the dependency of the histogram on the warping parameters is introduced via a feature kernel. This fact helps us to overcome several limitations of kernel-based methods. The target is represented by a collection of patch-based histograms, thus retaining spatial information. A robust statistics scheme assigns weights to the different patches, rendering the algorithm robust to partial occlusions and appearance changes. We present the algorithm for 1-D histograms (e.g. gray-scale), however extending the algorithm to handle higher dimensional histograms (e.g. color) is straightforward. Our method applies to any warping transformation that forms a group, and to any smooth feature. It has the same asymptotic complexity as the original inverse compositional template matching algorithm. We present experimental results which demonstrate the robustness of our algorithm, using only gray-scale histograms.