Generation of high-quality depth maps using hybrid camera system for 3-D video

In this paper, we present a hybrid camera system combining one time-of-flight depth camera and multiple video cameras to generate multi-view video sequences and their corresponding depth maps. In order to obtain the multi-view video-plus-depth data using the hybrid camera system, we capture multi-view videos using multiple video cameras and a single view depth video with the depth camera. After performing a three-dimensional (3-D) warping operation to obtain an initial depth map at each viewpoint, we refine the initial depth map using segment-based stereo matching. To reduce mismatched depth values along object boundaries, we detect the moving objects using color difference between frames and extract occlusion and disocclusion areas with the initial depth information. Finally, we recompute the depth value of each pixel in each segment using pairwise stereo matching with a proposed cost function. Experimental results show that the proposed hybrid camera system produces multi-view video sequences with more accurate depth maps, especially along the boundary of objects. In addition, it is suitable for generating more natural 3-D views for 3-D TV than previous works..

Research highlights► We generate high-definition multi-view depth maps using the hybrid camera system. ► The depth information acquired by a depth camera is used as the initial depth maps. ► We perform segment-based pairwise stereo matching for the redefined segments. ► Depth refinement for each segment is conducted using the proposed cost function.