Perceptual based SAO rate-distortion optimization method with a simplified JND model for H.265/HEVC

In the latest H.265/High Efficiency Video Coding (HEVC) standard, the sample adaptive offset (SAO) filter technique is adopted to improve the quality of the reconstructed video. But so far, the research efforts related to the SAO optimization to date have mainly centered on objective rate-distortion (R-D) performance without considering its visual quality. In this paper, the human visual characteristics (represented by a JND model) are introduced into the SAO optimization process for the first time, and a new human visual perceptual-based SAO R-D optimization method, referred to as P-SAO, for H.265/HEVC is proposed. Simultaneously, considering the SAO R-D optimization in H.265/HEVC is complex and in order to use the JND model more effectively and to minimize the calculation time of the introduced JND in SAO, a simplified JND model is proposed based on a modified Sobel operator. Experimental results show that compared with the latest JND model in pixel domain, the proposed JND model can achieve similar subjective quality with significantly reduced computational complexity (i.e., an average processing time reduction of 89.35%). Compared with the original SAO R-D method in the reference software model of the H.265/HEVC, the P-SAO method can achieve better image subjective quality with performance gain of up to 0.2505 dB in terms of ΔPSPNR without comprising the R-D performance in ΔPSNR.