Illumination compensation for multi-view video based on layered histogram matching with depth information

Recently, 3-dimensional, stereo, and multi-view videos have been gathering attention for next-generation imaging services. However, they suffer from illumination mismatches caused by differences in camera positions for different viewpoints and imperfect camera calibration. Multi-view videos are more susceptible to these problems. They usually decrease the performance of further processes such as data compression.We propose several component techniques to reduce the effects of these problems, built on the basis of time-constant histogram matching methods. Our method consists of a pre-processing filter to increase the pixel correlation; a color conversion technique to consider the value range of each color component; an image separation technique to separate an image into several layered sub-images using corresponding depth information; and a technique to group the temporal frames as appropriate processing units for illumination compensation.To show the performance of the proposed techniques, we conduct experiments that apply the methods to several multi-view video test sequences for illumination compensation, which are followed by a data compression process using a standard tool. In these experiments we compose the proposed techniques into four combinations on the basis of the time-constant histogram matching method. The experimental results show that all four combinations increased the image quality more than 2 dB in PSNR at the same bit-rate, or decreased the bit-rate by about 20% at the same image quality in comparison to the JMVM, which includes an illumination compensation technique. In comparison with previous studies, all four combinations showed better performance. Thus, we can conclude that each of the proposed techniques contributes to illumination compensation and multi-view video coding.