Visible and infrared image registration using trajectories and composite foreground images

The registration of images from multiple types of sensors (particularly infrared sensors and visible color sensors) is a step toward achieving multi-sensor fusion. This paper proposes a registration method using a novel error function. Registration of infrared and visible color images is performed by using the trajectories of moving objects obtained using background subtraction and simple tracking. The trajectory points are matched using a RANSAC-based algorithm and a novel registration criterion, which is based on the overlap of foreground pixels in composite foreground images. This criterion allows performing registration when there are few trajectories and gives more stable results. Our method was tested and its performance quantified using nine scenarios. It outperforms a related method only based on trajectory points in cases where there are few moving objects.

Graphical AbstractFigure optionsDownload high-quality image (129 K)Download as PowerPoint slideResearch Highlights
► Registration of IR and Visible videos using trajectories of moving objects.
► Registration criterion is the number of overlapping foreground pixels.
► Our method performs even if there are few trajectories.
► Using composite foreground images improves registration accuracy.
► Appropriate for methods performing image fusion using blob contours or edges.