Joint trajectory tracking and recognition based on bi-directional nonlinear learning

Motion trajectory is one of the most important cues for tracking and behavior recognition and can be widely applied to numerous fields. However, it is a difficult problem to directly model the spatio-temporal variations of trajectories due to their high dimensionality and nonlinearity. In this paper, we propose a joint trajectory tracking and recognition algorithm by combining a generative model derived from a bi-directional deep neural network (called “autoencoder”) into a Bayesian estimation framework. The “autoencoder” network embeds high-dimensional trajectories into a two-dimensional plane based on a peculiar training rule and learns a trajectory generative model by its inverse mapping. A set of plausible trajectories can be generated by the trajectory generative model. In the tracking process, the samples from the plausible trajectory set are weighted by a mixed likelihood and are resampled to obtain the target state estimation at each time step in spirit of the particle filtering. The trajectory identity is inferred by evaluating the improved Hausdorff distance between the estimated trajectory up to now and the truncated reference trajectories. Moreover, the trajectory recognition results are also used to guide the trajectory tracking for the next time. The experiments on tracking and recognizing handwritten digits show that the proposed approach can achieve both robust tracking and exact recognition in background clutter and partial occlusion.