Objective quality assessment in free-viewpoint video production

This paper addresses the problem of objectively quantifying accuracy in free-viewpoint video production. Free-viewpoint video makes use of geometric scene reconstruction and renders novel views using the appearance sampled in multiple camera images. Previous work typically adopts an objective evaluation of geometric accuracy against ground-truth data or a subjective evaluation of visual quality in view synthesis. We consider two production scenarios, human performance capture in a highly constrained studio environment and sports production in a large-scale external environment. The accuracy of scene reconstruction is typically limited and absolute geometric accuracy does not necessarily reflect the quality of free-viewpoint rendering. A framework is introduced to quantify error at the point of view synthesis. The approach can be applied as a full-reference metric to measure fidelity to a ground-truth image or as a no-reference metric to measure the error in rendering. The framework is applied to a data set with known geometric accuracy and a comparison is presented for studio based and sports production scenarios.