Efficient and robust multi-template tracking using multi-start interactive hybrid search

•We present an efficient and robust multi-template based visual object tracking method.•The target is represented by two heterogeneous time-varying templates using Gaussian functions.•The template is adaptively updated so that it can address both short- and long-term appearance variations.•Target is located by an interactive multi-start hybrid search.•Localization method consists of a sampling- and gradient-based algorithm in a unified probabilistic framework.

This paper presents an efficient, accurate, and robust template-based visual tracker. In this method, the target is represented by two heterogeneous and adaptive Gaussian-based templates which can model both short- and long-term changes in the target appearance. The proposed localization algorithm features an interactive multi-start optimization process that takes into account generic transformations using a combination of sampling- and gradient-based techniques in a unified probabilistic framework. Both the short- and long-term templates are used to find the best location of the target, simultaneously. This approach further increased both the efficiency and accuracy of the proposed tracker. The contributions of the proposed tracking method include: (1) Flexible multi-model target representation which in general can accurately and robustly handle challenging situations such as significant appearance and shape changes, (2) Robust template updating algorithm where a combination of tracking time step, a forgetting factor, and an uncertainty margin are used to update the mean and variance of the Gaussian functions, and (3) Efficient and interactive multi-start optimization which can improve the accuracy, robustness, and efficiency of the target localization by parallel searching in different time-varying templates. Several challenging and publicly available videos have been used to both demonstrate and quantify the superiority of the proposed tracking method in comparison with other state-of-the-art trackers.