Video super-resolution with registration-reliability regulation and adaptive total variation

• An registration reliability regulated data-fidelity constraint.
• A nonlocal similar block based structure tensor estimation.
• A content adaptive total variation regulation.

In super-resolution that constructs a high-resolution (HR) image from a set of low-resolution (LR) reference images, it is crucial to align the LR reference images in order to efficiently exploit the pixels therein. However, due to the existence of complex local motion, ideal registration is difficult to acquire. In this paper, we present a robust video super-resolution scheme with registration-reliability regulation and content adaptive total variation regularization, which make the scheme resilient to registration failures. In order to handle ill-registered pixels, we propose a registration-reliability regulated data-fidelity term, which assigns smaller weights to the pixels with larger locally-averaged registration residuals. In addition, a content adaptive total variation based on structure tensor, which is used to estimate image local structures, is proposed to regularize the super-resolved images. The structure tensor is derived not only from the gradients of local patches but also the nonlocal similar patches. Experimental results show that the proposed scheme can remarkably improve both the objective and subjective quality of the video super-resolution results.