Robust residual error consistent tracker with ranking mechanism

•A residual error term is used to constraint the objective function.•A kind of residual error score is extracted from residual errors.•A new ranking mechanism is designed to fuse the likelihood information.

In the paper, we propose a novel structural local sparse representation based residual error consistent ranking tracker. In our tracker, candidate targets are linearly combined by using the structural local sparse appearance model. To encourage temporal consistency, a residual error consistency term is designed to constraint the objective function of sparse representation. Based on the objective function, the similarity information is extracted from both coefficients and residual errors of sparse coding. For extracting similarity information from coefficients, the alignment-pooling algorithm is applied to obtain pooled features. For extracting similarity information from residual errors, we develop a residual error score. For different natures of residual error scores and pooled features, a ranking mechanism is proposed to fuse them. The dictionary updating scheme uses the ranking results of the predicted targets to determine which of them are collected for updating. Our tracker performs favorably against 6 state-of-the-art trackers on 18 challenging sequences.