Large-scale outdoor 3D reconstruction on a mobile device

This paper presents an approach for reconstructing large-scale outdoor scenes through monocular motion stereo at interactive frame rates on a modern mobile device (Google Project Tango Development Kit Tablet). The device's fisheye camera enables a user to reconstruct large scenes in only a few minutes by simply walking through the scene. We utilize the device's GPU to compute depth maps via plane sweep stereo. In contrast to reconstructing small objects, we observe that in large-scale scenarios using motion stereo, free-space measurements are less effective for suppressing outliers due to limited possibilities for camera placement and an unbounded reconstruction volume. Furthermore, the outlier ratio in depth maps from stereo matching is much higher compared to images from depth sensors. Consequently, we propose a set of filtering steps to detect and discard unreliable depth measurements. The remaining parts of the depth maps are then integrated into a volumetric representation of the scene using a truncated signed distance function. Ours is the first method to enable live reconstruction of large outdoor scenes on a mobile device. We extensively evaluate our approach, demonstrating the benefit of rigorously filtering depth maps.