Super resolution of laser range data based on image-guided fusion and dense matching

Low-cost/real-time laser range scanner is becoming one of the dominant tools in acquiring accurate 3D point clouds for many smart applications (e.g. automated driving), while the low point density is often the limiting factor for acquiring fine-scale information. On the other hand, stereo/multi-stereo images offer considerably higher-resolution 3D data with low cost, while the image-derived point clouds generally have a higher level of uncertainty. In order to generate accurate, dense point clouds at a low cost, this paper explores a complementary data fusion of the low-resolution-high-accuracy laser range data and the high-resolution (high-res) images, and proposes a super resolution method of laser range data through a novel dense matching framework. In general, we formulate the super resolution as maximizing a posteriori - Markov random field (MAP-MRF) problem in a constrained matching framework, where a two-step strategy is introduced to remove partial inconsistencies between laser range data and images, and the confidence of the high accuracy laser points are propagated through a uniquely designed path in the high-resolution image space, such that a global dense matching algorithm can be externally constrained to yield an accurate, dense and high-fidelity point clouds. We compared the experiment results of the proposed method with the original laser range data and other two super resolution methods of laser range data under aerial, terrestrial, and indoor scenarios. These all demonstrate that the proposed method is capable of producing sub-pixel accuracy, high-fidelity point clouds, even though the density of laser range data is considerably low (hundreds of times lower than the image resolutions).