Fully pipelined FPGA-based architecture for real-time SIFT extraction

•A fully pipelined FPGA-based architecture for real-time SIFT detector and descriptor is proposed.•Each descriptor vector is extracted in one clock cycle, i.e. 46 ns considering Cyclone IV technology.•The overall frame rate is independent of the number of detected features.•Measures of response, repeatability and matching capability are maintained very high.•Efficient resource usage allows the architecture to be hosted in a mid-range FPGA device.

Image feature extraction constitutes a fundamental task in robotic vision applications. Scale-Invariant Feature Transform (SIFT) has been widely used as a robust method for detecting and matching features. Nevertheless, SIFT algorithm is computationally demanding and its implementation in an embedded system requires a subtle approach. In this paper, an optimized and fully pipelined architecture is proposed for real-time detection of SIFT keypoints and extraction of SIFT descriptors. The system is suitable to target robotic vision applications and it is pipelined on pixel basis. The architecture is hosted in a medium-scale Cyclone IV FPGA device clocked at 21.7 MHz and is capable of extracting a feature with its descriptor at every clock cycle, i.e. in 46 ns. This processing speed is independent of the number of features detected in the input image and it therefore represents a very high SIFT throughput, adequate for the most demanding SIFT-based robotic applications. The system can process 70 fps in VGA resolution, while it keeps power dissipation at low levels. Moreover, the proposed implementation achieves high response and repeatability values and its matching ability is directly comparable with floating point software-based SIFT implementations. Design details are given for the combinational and RAM-based circuits forming the SIFT datapath.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide