RSMAT: Robust simultaneous modeling and tracking

This paper describes a robust on-line appearance modeling and tracking method, based on simultaneous modeling and tracking (SMAT). The appearance model is defined by a series of clusters, built in a video sequence using previously encountered samples. This model is used to search for the corresponding element in the following frames. Three alternative incremental clustering methods are proposed to increase the robustness and description capabilities of the model. The proposal is evaluated on an application of face tracking for driver monitoring. The test set comprises sequences of drivers recorded outdoors and in a truck simulator, which contain examples of occlusions and self-occlusions, as well as illumination changes. The performance is evaluated and compared with that of the original SMAT proposal and the recently presented stacked trimmed active shape model (STASM). Our proposal shows better results than the original SMAT and similar fitting error levels to STASM, with much faster execution times and better robustness to self-occlusions.

Research highlights► Three incremental clustering methods show improved robustness and description capabilities on image patches compared to SMAT. ► Tests on sequences recorded in a moving vehicle show lower fitting error and tracking losses. ► Short computing times with mean rate of over 100 fps.