A generalised framework for saliency-based point feature detection

•A novel multi-modal histogram-based salient point detector is proposed.•The general formulation allows salient point detection in both images and 3D data.•The derivative-based approach improves upon existing intensity based 2D approaches.•The 3D detector naturally operates on both the geometry and texture of 3D scenes.•Increased 2D-2D and 2D-3D repeatability demonstrated on indoor and outdoor scenes.

Here we present a novel, histogram-based salient point feature detector that may naturally be applied to both images and 3D data. Existing point feature detectors are often modality specific, with 2D and 3D feature detectors typically constructed in separate ways. As such, their applicability in a 2D-3D context is very limited, particularly where the 3D data is obtained by a LiDAR scanner. By contrast, our histogram-based approach is highly generalisable and as such, may be meaningfully applied between 2D and 3D data. Using the generalised approach, we propose salient point detectors for images, and both untextured and textured 3D data. The approach naturally allows for the detection of salient 3D points based jointly on both the geometry and texture of the scene, allowing for broader applicability. The repeatability of the feature detectors is evaluated using a range of datasets including image and LiDAR input from indoor and outdoor scenes. Experimental results demonstrate a significant improvement in terms of 2D-2D and 2D-3D repeatability compared to existing multi-modal feature detectors.