Real-time tracking-by-learning with high-order regularization fusion for big video abstraction

•By considering the object motion is an essential factor that not only needs to be taken care of while sampling during learning, we incorporate the motion information into an online tracking framework with an effective optimization process of the collaborative learning.•we model the object motion as a high order regularization, which is composed of a graph-based bidirectional trace regularizer and a kernelized confidence prediction regularizer.•For implementing with the abruption motion estimation and budget maintenance, the proposed tracking has demonstrated its advantages in both accuracy and efficiency, which also showed its practicability for big video abstraction applications.

Visual tracking is a key technique used by video abstraction to achieve efficient post-analysis for big video surveillance. In order to tackle the problem of constantly changing scenarios during online tracking, additional factors such as motion can be incorporated by utilizing a fusion strategy to improve the final performance. Unfortunately, straightforward output fusion is difficult to be synchronized due to the diversity in model regression. Therefore, a widely cited problem for learning based fusion is to incorporate regularizers for label assignment of unlabeled samples, which is one of the major research focuses on semi-supervised learning. In this paper, a novel tracking strategy based on semi-supervised learning with high order regularization fusion has been proposed. It employs two different types of regularizers to achieve more accurate label assignment based on kernelized confidence prediction and graph-based bi-directional trace from motion. The computation of the proposed tracker takes advantage of the unique feature of circulant matrix in Fourier domain and integral patterns, and thus can be readily implemented for real-time processing, even without any code optimization. Via a dynamic budget maintenance for model updating, the proposed tracking method demonstrated to outperform most state-of-art trackers on challenging benchmark videos with a fixed parameter configuration.