Human visual system inspired multi-modal medical image fusion framework

Multi-modal medical image fusion, as a powerful tool for the clinical applications, has developed with the advent of various imaging modalities in medical imaging. The main motivation is to capture most relevant information from sources into a single output, which plays an important role in medical diagnosis. In this paper, a novel framework for medical image fusion based on framelet transform is proposed considering the characteristics of human visual system (HVS). The core idea behind the proposed framework is to decompose all source images by the framelet transform. Two different HVS inspired fusion rules are proposed for combining the low- and high-frequency coefficients respectively. The former is based on the visibility measure, and the latter is based on the texture information. Finally, the fused image is constructed by the inverse framelet transform with all composite coefficients. Experimental results highlight the expediency and suitability of the proposed framework. The efficiency of the proposed method is demonstrated by the different experiments on different multi-modal medical images. Further, the enhanced performance of the proposed framework is understood from the comparison with existing algorithms.

► This paper presents a novel multi-modal medical image fusion framework for better diagnosis. ► The proposed framework relies on framelet transform and human visual system characteristics. ► Two new fusion rules are proposed to fuse low- and high-frequency bands. ► The efficiency of the framework is highlighted by different experiments on CT/MRI and PET/MRI images. ► The superiority of the framework is achieved by the comparison with state-of-art algorithms.