Novelty detection in human tracking based on spatiotemporal oriented energies

•Exploiting SOE features for modeling occlusion and detecting novelties in videos.•A “Bayesian model” to determine various states of the target during the tracking.•A “Novelty Detection” method to prevent template corruption in occlusion and drift.•Changing the tracking strategy in ‘Full Occlusion’ to a ‘predictive tracking’ method.•Thorough evaluation in occlusion scenarios on public surveillance datasets.

Integrated analysis of spatial and temporal domains is considered to overcome some of the challenging computer vision problems such as ‘Dynamic Scene Understanding’ and ‘Action Recognition’. In visual tracking, ‘Spatiotemporal Oriented Energy’ (SOE) features are successfully applied to locate the object in cluttered scenes under varying illumination. In contrast to previous studies, this paper introduces SOE features for occlusion modeling and novelty detection in tracking. To this end, we propose a Bayesian state machine that exploits SOE information to analyze occlusion and identify the target status in the course of tracking. The proposed approach can be seamlessly merged with a generic tracking system to prevent template corruption (for example when the target is occluded). Comparative evaluations show that the proposed approach could significantly improve the performance of a generic tracking system in challenging occlusion situations.