Line segment matching and reconstruction via exploiting coplanar cues

This paper introduces a new system for reconstructing 3D scenes from Line Segments (LS) on images. A new LS matching algorithm and a novel 3D LS reconstruction algorithm are incorporated into the system. Two coplanar cues that indicates image LSs are coplanar in physical (3D) space are extensively exploited in both algorithms: (1) adjacent image LSs are coplanar in space in a high possibility; (2) the projections of coplanar 3D LSs in two images are related by the same planar homography. Based on these two cues, we efficiently match LSs from two images firstly in pairs through matching the V-junctions formed by adjacent LSs, and secondly in individuals by exploiting local homographies. We extract for each V-junction a scale and affine invariant local region to match V-junctions from two images. The local homographies estimated from V-junction matches are used to match LSs in individuals. To get 3D LSs from the obtained LS matches, we propose to first estimate space planes from clustered LS matches and then back-project image LSs onto the space planes. Markov Random Field (MRF) is introduced to help more reliable LS match clustering. Experiments shows our LS matching algorithm significantly improves the efficiency of state-of-the-art methods while achieves comparable matching performance, and our 3D LS reconstruction algorithm generates more complete and detailed 3D scene models using much fewer images.