Gradient-domain-based enhancement of multi-view depth video

Multi-view depth is an emerging and attractive 3D representation in recent years, which acts a significant role in rendering numerous viewing angles from a small number of given input views. The performance of those depth-based 3D video applications is strongly dependent on the quality of multi-view depth. However, because of the limitations of depth acquisition and estimation, the quality of multi-view depth suffers from artifacts in spatial dimension and inconsistency in both the temporal and inter-view dimensions. In this paper, we propose a gradient-domain based enhancement method for multi-view depth. Being different from the traditional enhancement methods which intended to process one or two above dimensions, our proposal exploits the coherence of both temporal and inter-view dimensions in addition of the spatial one. It is very challenging to obtain the stable characteristics in multi-dimensions. To solve this problem, we propose to investigate the characteristics in the gradient domain rather than the intensity domain. The enhanced multi-view depth is obtained through the minimization of energy function under the constraint of a joint gradient field (JGF), which is estimated from multiple dimensions through motion estimation and geometric mapping. Therefore, the enhanced multi-view depth is a global optimization in multiple dimensions that ensures the consistency in temporal, inter-view and spatial domains. Furthermore, we also propose an enhancement structure to indicate the process order of depth frames. The experimental results suggest that the proposed method can enhance multi-view depth with desired sharpness and consistency.