Similarity-based multimodality image fusion with shiftable complex directional pyramid

For multimodality images, a novel fusion algorithm based on the shiftable complex directional pyramid transform (SCDPT) is proposed in this paper. As well, with the aid of the structural similarity (SSIM) index, a ‘similarity-based’ idea is employed to distinguish regions with ‘redundant’ or ‘complementary’ information between source imagers before the SCDPT coefficients are merged. A ‘weighted averaging’ scheme for regions with ‘redundant’ information and a ‘selecting’ scheme for regions with ‘complementary’ information are then employed, respectively. When merging the low-pass subband coefficients, the SSIM index in spatial domain (SP-SSIM) is employed as similarity measure, and three types of regions are thus determined. Especially, for regions with similar intensity values but different intensity changing directions between source images, a ‘selecting’ scheme based on gradient and energy is proposed. When merging the directional band-pass subband coefficients, the SSIM index in complex wavelet domain (CW-SSIM) is employed as similarity measure. With the CW-SSIM index, not only the magnitude information but also the phase information of SCDPT coefficients can be employed. Compared to the traditional energy matching (EM) index based fusion methods, the proposed method can better deal with ‘redundant’ and ‘complementary’ information of source images. In addition, because of the shift-invariance of the SCDPT and the CW-SSIM index, the proposed fusion algorithm performs well even if the input images are not well registered. Several sets of experimental results demonstrate the validity and feasibility of the proposed method in terms of both visual quality and objective evaluation.

► Magnitude information and phase information of the SCDPT are employed.
► Regions with redundant or complementary information are distinguished and merged.
► Regions containing borders or edges are particularly merged.
► The fusion method is with generality and good robustness on mis-registration.