Adaptive on-line similarity measure for direct visual tracking

•We present an adaptive metric for measuring the similarity of a target for the purpose of visual tracking.•This metric assigns a robust weight to each matching error based on the error type.•A histogram-based classifier is learned on-line to determine the error type.•The proposed robust metric dynamically adapts to the actual appearance changes by tuning its parameters.

This paper presents an on-line adaptive metric to estimate the similarity between the target representation model and new image received at every time instant. The similarity measure, also known as observation likelihood, plays a crucial role in the accuracy and robustness of visual tracking. In this work, an L2-norm is adaptively weighted at every matching step to calculate the similarity between the target model and image descriptors. A histogram-based classifier is learned on-line to categorize the matching errors into three classes namely i) image noise, ii) significant appearance changes, and iii) outliers. A robust weight is assigned to each matching error based on the class label. Therefore, the proposed similarity measure is able to reject outliers and adapt to the target model by discriminating the appearance changes from the undesired outliers. The experimental results show the superiority of the proposed method with respect to accuracy and robustness in the presence of severe and long-term occlusion and image noise in comparison with commonly used robust regressors.