Visible and infrared image registration in man-made environments employing hybrid visual features

We present a new method to register a pair of images captured in different image modalities. Unlike most of existing systems that register images by aligning single type of visual features, e.g., interest point or contour, we try to align hybrid visual features, including straight lines and interest points. The entire algorithm is carried out in two stages: line-based global transform approximation and point-based local transform adaptation. In the first stage, straight lines derived from edge pixels are employed to find correspondences between two images in order to estimate a global perspective transformation. In the second stage, we divide the entire image into non-overlapping cells with fixed size. The point having the strongest corner response within each cell is selected as the interest point. These points are transformed to other image based on the global transform, and then used to bootstrap a local correspondence search. Experimental evidence shows this method achieves better accuracy for registering visible and long wavelength infrared images/videos as compared to state-of-the-art approaches.

► We register infrared and visible images employing the combination of straight lines and feature points.
► We start from a global transformation, but end up with a local transformation computation.
► This global transformation is used to bootstrap a more accurate, locally adaptive transformation.
► Straight lines derived from boundaries of objects can easily be extracted and matched.