Accurate spatio-temporal reconstruction of missing data in dynamic scenes

•The proposed frames subtraction is closed to the on-the-fly background modeling.•The accurate localization of rigid objects by SURF tracking is applied.•Neural networks estimate smoothness, structural and isotropic properties of texture.•The tile reconstruction method uses a single texton, a line, or a field of textons.

In this paper, the accurate method for texture reconstruction with non-desirable moving objects into dynamic scenes is proposed. This task is concerned to editor off-line functions, and the main criteria are the accuracy and visibility of the reconstructed results. The method is based on a spatio-temporal analysis and includes two stages. The first stage uses a feature points tracking to locate the rigid objects accurately under the assumption of their affine motion model. The second stage involves the accurate reconstruction of video sequence based on texture maps of smoothness, structural properties, and isotropy. These parameters are estimated by three separate neural networks of a back propagation. The background reconstruction is realized by a tile method using a single texton, a line, or a field of textons. The proposed technique was tested into reconstructed regions with a frame area up to 8–20%. The experimental results demonstrate more accurate inpainting owing to the improved motion estimations and the modified texture parameters.