Fusion of multispectral and panchromatic images via sparse representation and local autoregressive model

•Pan-sharpening is the fusion of multispectral image and panchromatic image.•We model pan-sharpening into an image restoration problem.•Sparsity and local autoregressive model regularizations are used in the method.•Two dictionaries are constructed using the K-SVD algorithm.•Experiments prove the effectiveness of the proposed method.

In this paper, a novel model-based pan-sharpening method via sparse representation and local autoregressive (AR) model is proposed. To recover the high-resolution multispectral (HRMS) image from the observed images, we impose sparsity prior on the unknown HRMS image in the restoration model. The quality of the recovered HRMS image depends on the employed sparse domain. Hence, a new sparse representation model for the HRMS image is constructed, in which we suppose that the low-frequency and high-frequency components of the HRMS image can be sparsely represented by a spectral dictionary and a spatial-detail dictionary respectively. The spectral dictionary and spatial-detail dictionary are learned from the source images: low-spatial-resolution multispectral (LRMS) image and high-spatial-resolution panchromatic (HRP) image adaptively. Additionally, local autoregressive (AR) model is employed to improve the spatial structure of the HRMS image patch. Firstly, a set of AR model parameters are learned from the PAN image patches. Then, the local spatial structure of a given HRMS image patch is regularized by an AR model with the learned parameters. By solving the l1 -norm optimization problem, the HRMS image can be well reconstructed. Experiments are carried out on very high-resolution QuickBird and GeoEye-1 images. In the simulated and real experiments, our proposed method demonstrates its good performance in terms of visual analysis and quantitative evaluation.