Layered light-field rendering with focus measurement

This paper introduces a new image-based rendering method that uses input from an array of cameras and synthesizes high-quality free-viewpoint images in real-time. The input cameras can be roughly arranged, if they are calibrated in advance. Our method uses a set of depth layers to deal with scenes with large depth ranges, but does not require prior knowledge of the scene geometry. Instead, during the on-the-fly process, the optimal depth layer is automatically assigned to each pixel on the synthesized image by using our focus measurement scheme. We implemented the rendering method and achieved nearly interactive frame rates on a commodity PC. This paper also discusses the focus measurement scheme in both spatial and frequency domains. The discussion in the spatial domain is practical since it can be applied for arbitrary camera arrays. On the other hand, the frequency domain analysis is theoretically interesting since it proves that a signal-processing theory is applicable to the depth assignment problem.