Detection based object labeling of 3D point cloud for indoor scenes

While much exciting progress is being made in 3D reconstruction of scenes, object labeling of 3D point cloud for indoor scenes has been left as a challenge issue. How should we explore the reference images of 3D scene, in aid of scene parsing? In this paper, we propose a framework for 3D indoor scenes labeling, based upon object detection on the RGB-D frames of 3D scene. First, the point cloud is segmented into homogeneous segments. Then, we utilize object detectors to assign class probabilities to pixels in every RGB-D frame. After that, the class probabilities are projected into the segments. Finally, we perform accurate inference on a MRF model over the homogeneous segments, in combination with geometry cues to output the labels. Experiment on the challenging RGB-D Object Dataset demonstrates that our detection based approach produces accurate labeling and improves the robustness of small object detection for indoor scenes.