An interactive method for the image alignment problem based on partially supervised correspondence

•Partially supervised image alignment and point correspondence estimation.•Current automatic image registration fails on extremely different images.•With little human interaction, correct correspondences are found.•Our method shows good results in extremely different pairs of images.

In a robotic scenario in which robots are localised through the images they obtain, it is usual that the current image registration techniques fail due to large differences between images, occlusions, shadows and so on. In these cases a human can interact on the computer vision system to help it to find a preliminary mapping between some salient points. We present a method to solve the alignment problem between two images in which a human (or oracle) imposes a partial correspondence between salient points extracted from the images through a Human–Machine Interface and the Interactive Correspondence Method computes a new image alignment and a set of point correspondences. Practical evaluation shows that in two human interactions our method finds an almost optimal alignment and correspondence set between images that any current completely automatic method is able to obtain. Moreover, we demonstrate that in situations that there is a large difference between images, adding interactivity is the only method that achieves an acceptable result.