Multi-scale decomposition based fusion of infrared and visible image via total variation and saliency analysis

Infrared (IR) images are the thermal radiation of the objects which can record different object information of a scene. Visible images are the optical information of a scene and contain a lot of details. They can often provide complementary information for a scene in a fusion image, which is more suitable for both human vision and machine perception. Multi-scale decomposition (MSD), which has the advantage of extracting characteristics at different scales, is one of the most widely used fusion strategies. However, many traditional MSD based methods ignore the different imaging characteristics of IR and visible images. These methods use the same representations for the source images, which negatively impact the fused image. We propose a new MSD based fusion method with total variation minimization to overcome these drawbacks. Our method consists of three steps: decompose the source images with Gaussian filter to obtain base and detail layers; adopt different combination rules to fuse the base and detail layers; and reconstruct the fused image by fusing the combined base and detail layers. Our method can preserve the thermal radiation and details from the source images using different representations at different layers. We compare our combination rules with two classic rules and our method with seven state-of-the-art methods using qualitative and quantitative tests. Experimental results indicate that our rules outperform other rules and our method is performs better than the other seven methods.