RPR-SORS: Real-time photorealistic rendering of synthetic objects into real scenes

This paper presents a review of the Photorealistic Augmented Reality field and proposes a solution for interactively rendering virtual objects into dynamic real scenes in a photorealistic way. This solution features a rendering pipeline that comprises techniques regarding illumination, reflectance model, shadowing, composition, and camera effects. The techniques are chained in a flexible way, allowing the user to choose which techniques are to be enabled. An environment map generation procedure was developed and allows virtual objects to exhibit coherent effects such as color bleeding and specular reflection, even when the real objects are moved. The range of materials that can be rendered was widened by extending Lafortune's Spatial BRDF. The implemented infrastructure is offered as an authoring toolkit that consists of an API and a material editor tool. The aim of this authoring toolkit is to increase development productivity of Photorealistic Augmented Reality applications. The proposed solution was evaluated by taking into account visual and performance metrics. It allowed consistent rendering of dynamic scenes and photorealistic materials. The frame rate obtained was suitable to Augmented Reality applications when there were few virtual objects in the scene.

Figure optionsDownload high-quality image (86 K)Download as PowerPoint slideHighlights
► We present a review of the photorealistic AR field.
► We propose a development toolkit for photorealistic AR applications.
► This toolkit comprises a rendering API and a material editor tool.
► It features illumination, reflectance, shadowing, composition, and camera effects.
► An experimental application was developed to evaluate the toolkit productivity.