Pixel-wise regression using U-Net and its application on pansharpening

Convolutional neural networks are widely used for solving image recognition and other classification problems in which the whole image is considered as a single object. In this paper, we take the pansharpening problem of remote sensing images as an example to discuss how to establish pixel-wise regression models using convolutional neural networks. In order to meet the requirements of pixel-wise analysis on both the localization accuracy and the abstraction ability of the regression process, a U-shaped architecture is applied in our study to construct the network model. By establishing direct connections between convolution layers at the front end and the back end of the network, image features corresponding to different resolution levels can be retained. Then a regression relationship between these multi-resolution image features and the target image pixel values can be obtained. Experimental results show that the proposed regression model can effectively accomplish pansharpening, with better performance in controlling geometric deformation and color distortion, as compared to some state of the art methods.