Model-based motion blur estimation for the improvement of motion tracking

•We introduce a new motion blur aware tracking approach.•Motion blur is considered as an additional information source.•Video deblurring is achieved by extracting information from all frames.•We evaluate our approach on ground truth data captured with a specialized setup.

Video tracking is an important task in many automated or semi-automated applications, like cinematic post production, surveillance or traffic monitoring. Most established video tracking methods fail or lead to an inaccurate estimate when motion blur occurs in the video, as they assume, that the object appears constantly sharp in the video. In this paper, we present a novel motion tracking method with explicit modeling of motion blur, estimating the continuous motion of a rigid 3-D object with known geometry in a monocular video as well as the sharp object texture. Instead of treating motion blur as a potential source of errors, we take advantage of it and consider motion blur as an additional information source, providing information about the motion of the tracked object during the exposure. In an analysis-by-synthesis approach we explicitly model the effects of motion blur reconstructing the captured frames, in order to accomplish a more accurate estimation. We design our algorithm to be capable to run in parallel on the GPU using the common rendering pipeline and considering each frame individually to handle also long videos. We tested our approach on both synthetic and real videos. In both cases, we achieve significant improvements of accuracy and reductions of frame reconstruction error compared to the estimated motion of a rigid body tracker, without motion blur handling.