View-spatial–temporal post-refinement for view synthesis in 3D video systems

• Propose a post-refinement method to improve the rendering quality of synthesized image.
• Propose hole filling and boundary artifacts removal technologies.
• View-spatial–temporal correlations of mutiview video are efficiently used to improve the rendering quality.
• A good trade-off between complexity and rendering quality is achieved in reference frame selection.
• Both subjective and objective qualities of rendered images are significantly improved.

Depth image based rendering is one of key techniques to realize view synthesis for three-dimensional television and free-viewpoint television, which provide high quality and immersive experiences to end viewers. However, artifacts of rendered images, including holes caused by occlusion/disclosure and boundary artifacts, may degrade the subjective and objective image quality. To handle these problems and improve the quality of rendered images, we present a novel view-spatial–temporal post-refinement method for view synthesis, in which new hole-filling and boundary artifact removal techniques are proposed. In addition, we propose an optimal reference frame selection algorithm for a better trade-off between the computational complexity and rendered image quality. Experimental results show that the proposed method can achieve a peak signal-to-noise ratio gain of 0.94 dB on average for multiview video test sequences when compared with the benchmark view synthesis reference software. In addition, the subjective quality of the rendered image is also improved.