Sequentially adaptive active appearance model with regression-based online reference appearance template

•An online reference template is generated for tracking drifts correction.•A correspondences map from Local feature to Landmark is learned via regression.•An online reference appearance model update strategy is designed.•The tracking accuracy is better than other state-of-the-art methods.

Statistically motivated approaches, such as the active appearance model (AAM), have been widely used for non-rigid objects registration and tracking. As an extension of AAM, sequential AAM (SAAM) was proposed, in which both an incremental updated component and a reference component were employed simultaneously in the fitting scheme. To make SAAM more adaptive to facial context variations during tracking, a regression-based online reference appearance model (ORAM) is presented to update the subject-specific appearance of the SAAM. The spatial map between scattered local feature correspondences and structured landmark correspondences is learned via Kernel Ridge Regression (KRR). Additionally, a shape deformation and appearance model evaluation strategies help to improve the accuracy and efficiency of the algorithm. The approach is experimentally validated by tracking face videos with improved fitting accuracy.