Block world reconstruction from spherical stereo image pairs

•We propose a 3D block-based scene reconstruction system using spherical images.•Facade alignment and cubic projection produce central-point perspective images.•Block representation is recovered based on optimal occupancy and physical stability.•Optional user interaction to constrain primitive reconstruction is provided.•Texture mapping from the original images gives a quick rendering of the scene.

We propose a block-based scene reconstruction method using multiple stereo pairs of spherical images. We assume that the urban scene consists of axis-aligned planar structures (Manhattan world). Captured spherical stereo images are converted into six central-point perspective images by cubic projection and façade alignment. Depth information is recovered by stereo matching between images. Semantic regions are segmented based on colour, edge and normal information. Independent 3D rectangular planes are constructed by fitting planes aligned with the principal axes of the segmented 3D points. Finally cuboid-based scene structure is recovered from multiple viewpoints by merging and refining planes based on connectivity and visibility. The reconstructed model efficiently shows the structure of the scene with a small amount of data.