Single image-based data-driven indoor scene modeling

•A new approach for 3D indoor scenes reconstruction from a single image is proposed.•The approach includes a novel feature encoding method for line drawings.•Relevance feedback for retrieval is provided to solve semantic gap between human and computer system.

With a single input indoor image (including sofa, tea table, etc.), a 3D scene can be reconstructed from a single image using an existing model library in three stages: image analysis, model retrieval and relevance feedback. In the image analysis stage, we obtain the object information from the input image using geometric reasoning technology combined with an image segmentation method. In the model retrieval stage, line drawings are extracted from 2D objects and 3D models by using different line rendering methods. We exploit various tokens to represent local features and then organize them together as a star-graph to show a global description. By comparing similarity among the encoded line drawings, models are retrieved from the model library. Also, for a better user experience, we add a relevance feedback stage following the retrieval stage. The Support Vector Machine method is used to conduct the feedback operation. After this stage, the retrieved models are in conformance with the image semantic. The 3D scene is then reconstructed. Experimental results show that, driven by the given model library, indoor scenes modeling from a single image could be achieved automatically and efficiently.

Graphical abstractFigure optionsDownload full-size imageDownload high-quality image (243 K)Download as PowerPoint slide