Image-based 3D scene reconstruction and exploration in augmented reality

Augmented Reality (AR) is an integrated technique of image processing and display system of complex information, which involves real-time computing, motion tracking, pattern recognition, image projection, database linking, feature extraction, and coordinate transformation. In this study, such techniques through Structure From Motion (SFM), Clustering Views for Multi-View Stereo (CMVS), Patch-based Multi-View Stereo (PMVS), and Poisson surface reconstruction were smoothly integrated into a 3D reconstruction system with comparative efficiency in computation. The system can be applied to regular images taken by amateur cameras, smart phones, tablet PCs, and other mobile devices, without need of a priori internal and external camera parameters. To verify the ability of the established reconstruction system, indoor and outdoor objects at various scales, such as a helmet (a small object), a corridor (an indoor medium object), an arbor (outdoor medium object), and a building (outdoor large object) were tested. Through tracking and registration, the reconstructed 3D models were loaded in an AR environment to facilitate displaying, interacting, and rendering that provides AR applications in construction design and management for better and qualitative communication in economical and handy ways.

► 3D reconstruction and AR tools were used for scene reconstruction and exploration.
► SFM, CMVS, PMVS, and surface reconstruction were integrated to establish 3D models.
► Through tracking and registration, the reconstructed 3D models were displayed in AR.