Embedded registration of visible and infrared images in real time for noninvasive skin cancer screening

We present an embedded system architecture that implements real-time multimodal registration to enable dual-camera spatio-temporal feature extraction in a skin cancer screening application. We test the system on a combination of visible and long-wave infrared image sequences, but it can be easily extended to setups operating in different sections of the spectrum. Image registration is performed by matching common features between each frame of a visible image to each frame of an infrared image sequence to estimate a projective transformation between them. The parameters of this transformation are estimated recursively on line with the video, thus enabling image registration in real time. The algorithm is implemented using a combination of embedded software and dedicated hardware units on a heterogeneous reconfigurable system-on-a-chip. The hardware performs feature detection and extraction, while the software estimates the transformation parameters and maps each visible video frame onto the infrared image coordinates. Implemented on an FPGA, our prototype runs at 540 frames per second with a 135 MHz clock, consumes 1.8 W and utilizes 29% and 54% of the logic and multiplier resources of the chip, respectively.