Motion-compensated frame interpolation using patch-based sparseland model

•We design a patch-based Bidirectional Motion Estimation (BME) to assign a unique Motion Vector (MV) for each patch by using optical-flow ME.•We construct the sparse model of the intermediate frame according to the MV of each pixel output by the BME module. This sparseland priori is formulated as a Maximum a Posteriori (MAP) estimation problem under the Bayesian framework, and this MAP estimation problem becomes the non-linear sparseland-prior reconstruction model.

Using patch-based sparseland model, we design a novel Motion-Compensated Frame Interpolation (MCFI) method. Optical-flow estimation is first adopted to generate a reliable Motion Vector Field (MVF) from the previous frame to the following frame. Then we use patch-based bidirectional motion estimation to generate a smooth Motion Vector Felid (MVF). In the last step, we extract patches from reference frames along the motion trajectories, and perform Principle Component Analysis (PCA) to generate dictionaries that code the patches with various local structures. The sparseland model becomes the prior knowledge of the intermediate frame to fuse into the Motion Compensated Interpolation (MCI) by maximum a posteriori (MAP) criterion. By iterative numerical computing, we solve this sparseland-prior reconstruction model, and acquire a sparsity-preferred interpolated frame. Experimental results show that our method outperforms the existing algorithms in both objective and subjective picture qualities, but it also introduces a high computational complexity in the meantime.