A rapid and accurate infrared image super-resolution method based on zoom mechanism

In view that the existing methods for infrared image super-resolution could not have good performance both in speed and accuracy simultaneously, with the advantage of deep learning method, this paper presents a rapid and accurate infrared image super-resolution method based on zoom mechanism. In the meantime, we design a novel network architecture. First, we take low-resolution images as network inputs directly, and employ a convolution layer to extract and represent features. Then we introduce the combination of a deconvolution layer and a pooling layer into the network, what we name as zoom mechanism. The zoom mechanism could not only enlarge and shrink obtained feature maps successively to extract features that are more sensitive to the results, but also implement the nonlinear mapping in a more effective way than other methods. Moreover, we depend on a sub-pixel convolution layer to realize the features map and images fusion in a single step. Subsequently, a plenty of natural images are utilized as the auxiliary training samples to obtain the pre-trained network. Finally, the pre-trained network is fine-tuned again by adding a few infrared images as objective training samples, which could make the network more suitable for infrared image super-resolution. The proposed approach is tested on both of natural and infrared images. It can achieve more satisfactory results both in objective criterion and subjective perspective compared to other state-of-the-art methods. In addition, it can process more than 24 images in size of 320 × 240 pixel per second. The experimental results show that the proposed method can not only generate images in higher quality, but also satisfy the requirement of real-time video super-solution.