Combining histogram-wise and pixel-wise matchings for kernel tracking through constrained optimization

•Pixel-wise matching is combined with histogram-wise matching.•Weight image considers both the foreground and background.•Weight matching maximizes likelihood ratio between foreground and background.•Template matching computes the trade-off between accuracy and robustness.•Pixel-wise similarity is optimized under the constraints of histogram-wise similarity.

In this paper, we propose a constrained optimization approach to improving both the robustness and accuracy of kernel tracking which is appropriate for real-time video surveillance due to its low computational load. Typical tracking with histogram-wise matching provides robustness but has insufficient accuracy, because it does not involve spatial information. On the other hand, tracking with pixel-wise matching achieves accurate performance but is not robust against deformation of a target object. To find the best compromise between robustness and accuracy, in our paper, we combine histogram-wise matching and pixel-wise template matching via constrained optimization problem. Firstly, we propose a novel weight image representing both the probability of foreground and the degree of similarity between the template and a candidate target image. The weight image is used to formulate an objective function for the histogram-wise weight matching. Then the pixel-wise matching is formulated as a constrained optimization problem using the result of the histogram-wise weight matching. In consequence, the proposed approach optimizes pixel-wise template similarity (for accuracy) under the constraints of histogram-wise feature similarity (for robustness). Experimental results show the combined effects, and demonstrate that our method outperforms recent tracking algorithms in terms of robustness, accuracy, and computational cost.