Fine registration of 3D point clouds fusing structural and photometric information using an RGB-D camera

•We address the problem of multiview alignment of 3D point clouds from RGB-D cameras.•We introduce a new cost function with dynamic weights to improve the ICP algorithm.•We include a novel outlier rejection method, based on structural and photometric cue.•Experimental results show the effectiveness of our method.

We address the problem of accurate and efficient alignment of 3D point clouds captured by an RGB-D (Kinect-style) camera from different viewpoints. While the Iterative Closest Point (ICP) algorithm has been widely used for dense point cloud matching, it is limited in its ability to produce accurate results in challenging scenarios involving objects that lack structural features and have significant camera view changes. In this paper, we introduce a new cost function with dynamic weights for the ICP algorithm to tackle this problem. It balances the significance of structural and photometric features with dynamically adjusted weights to improve the error minimization process. Our algorithm also includes a novel outlier rejection method, which adopts adaptive thresholding at each ICP iteration, using both the structural information of the object and the spatial distances of sparse SIFT feature pairs. The effectiveness of our proposed approach is demonstrated by experimental results from various challenging scenarios. We obtained superior registration accuracy than related previous methods, at the same time maintaining low computational requirements.