Robust tracking using visual cue integration for mobile mixed images

• Maximizing observation likelihood to optimize particle weights under reflections.
• Combining co-inference and maximum likelihood for visual cue integration.
• Co-inference is combined with SIR to avoid the particle degeneracy problem.
• Maximum likelihood selects the more reliable cue for co-inference fused state.
• Motion compensation for both layer separation and prediction of the particle filter.

The transmitted scene superposed with the reflected scene from a transparent surface leads to mixed images. Few methods have been devoted for tracking on mixed images while such images are ubiquitous in the real world. Thus, this paper proposes a robust single object tracking scheme for mixed images acquired by mobile cameras. Layer separation that decomposes mixed images extracts intrinsic dynamic layers before tracking. In order to make the tracker robust against camera motion, motion compensation is applied to both layer separation and prediction stage of the particle filter. To maximize the observation likelihood and thus optimize particle weights in the face of reflections, the proposed scheme combines sequential importance resampling (SIR) based co-inference and maximum likelihood for multi-cue integration. Experimental results show that the proposed scheme effectively improves tracking accuracy on mixed images with camera motion.