Color difference weighted adaptive residual preprocessing using perceptual modeling for video compression

•We investigate a method for video compression using a human visual system model.•We use color appearance, saliency, and spatial activity information from the scene.•The method has been implemented in the JM 18.0 H.264/AVC encoder reference software.•Significant improvements in compression are obtained without perceptible degradation.•The imperceptibility of the changes was confirmed through psychophysical evaluation.

In this paper, we investigate a method for selectively modifying a video stream using a color contrast sensitivity model based on the human visual system. The model identifies regions of high variance with frame-to-frame differences that are visually imperceptible to a human observer. The model is based on the CIE LAB color appearance model and the CIE ΔE94ΔE94 color difference formula, taking advantage of the nature of frame-based progressive video coding. The use of a color contrast sensitivity model alone was not sufficient. Therefore, it was important to also incorporate perceptual saliency and spatial activity information from the scene. The method has been implemented in the JM 18.0 H.264/AVC encoder reference software and resulted in an average gain of 37% in data compression without perceptible degradation of the video quality. As expected, the amount of compression improvement obtained is dependent on the type of video content being compressed. The imperceptibility of the changes was confirmed through psychophysical evaluation.