Multi-level visual tracking with hierarchical tree structural constraint

•Parts at three levels of granularity are utilized to represent the target.•Relationships between parts at all levels are modeled by the hierarchical tree.•Positions of all the parts are optimized jointly in a unified objective function.•The appearance model and the hierarchical tree structure are updated online.

Recently, part-based model has drawn much attention in visual tracking for its promising results in handling occlusion and deformation. However how to divide the target into parts and how to model the relationships between parts are still open problems. In this paper, we propose a robust tracker based on multi-level target representation and hierarchical tree structural constraint. The multi-level target representation models the target at three different levels: the bounding box (top) level, the superpixel (middle) level and the keypoint (bottom) level. The relationships between parts at all levels are modeled by the proposed hierarchical tree which includes intra-layer and inter-layer structural constraints. The positions of all the parts are optimized jointly in a unified objective function taking into account both the appearance similarity and the hierarchical tree structural constraint. The appearance model and the hierarchical tree structure are updated online to adapt to the changes of the target in both appearance and structure. Extensive experiments on various challenging video sequences demonstrate that the proposed method outperforms the state-of-the-art trackers significantly.