Jointly registering and fusing images from multiple sensors

• Jointly registering and fusing images from multiple sensors.
• Both ensemble registration and fusion can be treated as estimation problems.
• Both registration and fusion performances are improved.

In this paper, a novel approach is proposed for jointly registering and fusing a multisensor ensemble of images. Based on the idea that both groupwise registration and fusion can be treated as estimation problems, the proposed approach simultaneously models the mapping from the fused image to the source images and the joint intensity of all images with motion parameters at first, and then combines these models into a maximum likelihood function. The relevant parameters are determined through employing an expectation maximization algorithm. To evaluate the performance of the proposed approach, some representative image registration and fusion approaches are compared on different multimodal image datasets. The criterion, which is defined as the average pixel displacement from its true registered position, is used to compare the performances of registration approaches. As for evaluating the fusion performance, three fusion quality measures which are metric Qab/f, mutual information and average gradient are employed. The experimental results show that the proposed approach has improved performance compared to conventional approaches.