Patch-based spatio-temporal super-resolution for video with non-rigid motion

This paper presents a novel approach for spatio-temporal video super-resolution. Whereas the task of synthesizing high-frequency information on the spatial domain can be accomplished without introducing arbitrary priors on the image model (beyond the assumption of local cross-scale self-similarity), the high degree of temporal aliasing in standard-frame-rate video requires applying motion compensation in order to correctly interpolate the video sequence along the temporal axis. As the experimental results show, the proposed technique is capable of augmenting both the frame-rate of a video sequence (by any real up-converting factor) and its effective spatial resolution (by any rational magnification factor). The presented method is suitable for parallelized computing environments, such as GPUs, and provides an output quality comparable to state-of-the-art methods in both temporal interpolation and spatial super-resolution.

► Spatial and temporal video super-resolution with arbitrary, non-rigid motion.
► Spatial super-resolution achieved by exploiting cross-scale self-similarity.
► Temporal super-resolution accomplished by non-rigid motion compensation.
► GPU-friendly approximate solution for block matching data cost with regularization.
► Numerical and visual improvements with respect to the state-of-the-art.