Spatio-temporal super-resolution for multi-videos based on belief propagation

Aiming at improving spatio-temporal resolution of video for real-world applications, this paper focuses on the spatio-temporal super-resolution reconstruction algorithm for multiple asynchronous video sequences. The current multi-video super-resolution algorithms still have some weaknesses in application such as oversimplification of motion, unknown blurring and noise level. For further improvement, a Maximum Posterior Likelihood-Markov Random Field (MAP-MRF) based super-resolution reconstruction method is proposed and proved robust in achieving the real-world super-resolution imaging. The proposed method adopts weighted 3D neighborhood system (WNS) MAP-MRF model to accurately describe the spatial and temporal correlations between multiple video sequences. In order to improve the estimation accuracy of the motions for complex scenes, the improved Scale-Invariant Feature Transform (SIFT) Flow algorithm based on sparsity in wavelet domain is proposed, which can afford the large displacement, rotational movement and other complexities in asynchronous multi-video sequences. The Belief Propagation (BP) algorithm is applied to estimate the parameters of MAP-MRF model, such as motion vector and the super-resolution images in an iterative coarse-to-fine scheme. With the proposed algorithms mentioned above, MAP-MRF based super-resolution reconstruction method has better capabilities of edge sharpness and detailed texture preserving, and robustness of noise suppressing. The experimental result has confirmed the effectiveness of the proposed method under the practical conditions.