Multimodality image fusion by using both phase and magnitude information

In most of complex wavelet based fusion methods, only magnitude (or absolute value) of a complex coefficient is considered and phase information is neglected. However, more salient image features can be determined by the phase. In this paper, a multimodality image fusion algorithm is proposed with the shiftable complex directional pyramid transform (SCDPT), where phase and magnitudes of complex coefficients are jointly considered. Firstly, a novel similarity index (CCC-EM) is presented by combining the circular correlation coefficient (CCC) of relative phase angles and the traditional energy matching (EM) index. When bandpass directional subband coefficients are merged, the CCC-EM index is employed as the similarity measure and three types of regions between source images are determined for each bandpass directional subband. Then, based on some weights or salience measures, different fusion rules are designed for each type of regions. Especially, for regions with similarity in energy and positive or negative correlation relationship in relative phase, the weighted circular variance (WCV) of relative phase angles is employed. When lowpass subband coefficients are merged, the traditional structural similarity index is employed to distinguish different types of regions. For most of regions, the local energy of lowpass subband coefficients is employed as weights or salience measures. While for regions with similar intensity values but different intensity variation directions, an inter-scale based salience measure is defined by combining the local energy of the lowpass subband coefficients and the WCV of the coarsest bandpass directional subband coefficients. Several pairs of multimodality images are fused with the proposed methods. Fusion results demonstrate that the proposed fusion method can extract more salient features (not just in energy) from source images than some other complex wavelet based fusion methods. Especially, more phase information of source images can be preserved into the fused image, which makes the proposed fusion method with higher performance in spatial consistency.

► Magnitudes and phase of complex wavelet coefficients are both considered during the fusion process.
► For complex subband coefficients, a similarity measure is proposed with the EM index and CCC of relative phase.
► WCV of relative phase is employed as weight or salience when merging bandpass directional subband coefficients.
► An inter-scale based salience measure is defined for one type of regions in lowpass subband.
► The proposed method extracts more phase information from source images and performs better in spatial consistency.