Visual tracking with structural appearance model based on extended incremental non-negative matrix factorization

•An appearance model is built up with extended incremental non-negative matrix.•The discrimination of the encoding matrix is investigated.•A framework of visual tracking based on adaptive appearance model is established.•Performance of tracking is evaluated by quantitative and qualitative analysis.•Experiment results show superior performance in accuracy and robustness.

In this paper, we propose an appearance model based on extended incremental non-negative matrix factorization for visual tracking. With non-negative matrix factorization, each object image patch identified in a frame is regarded as a linear combination of a set of non-negative basis vectors. Then a feature space depicting the appearance of the object is spanned by the set of basis vectors. An encoding matrix encodes the image patches with the coding coefficient vectors. The discrimination of the coding coefficients is investigated. In order to search the image region for the object in a new frame, a group of image patches are sampled around the position of the object in the previous frame. A naïve Bayes classifier evaluates the confidence of each image patch as the candidate for the object image patch. Finally, the image patch with the maximal confidence is taken as the image region of the object. After identification of the object image patch, the mixing matrix and the classifier are updated incrementally. Experimental results show that our algorithm outperforms several state-of-the-art algorithms.