Quadtree-based non-local Kuan’s filtering in video compression

• We propose an adaptive loop filter to suppress quantization noise optimally.
• The proposed filter is performed on the transformed residual coefficients.
• Non-local parameter estimation and quantization constraint sets are unified.
• Quadtree data structure is used for optimal filtering control.
• Good balance is achieved between RD performance and computational complexity.

Transform coding has been widely used in video coding standards, such as H.264 advanced video coding (H.264/AVC) and high efficiency video coding (HEVC). But the coded video sequences suffer from annoying coding artifacts, such as blocking and ringing artifacts. In this paper, we propose the quadtree-based non-local Kuan’s (QNLK) filter to suppress the quantization noise optimally and improve the objective and subjective quality of the reconstructed frame simultaneously. The proposed filter takes advantage of the non-local Kuan’s (NLK) filter to restore the quantized signal in transform domain. Restored coefficients are then projected onto designed quantization constraint sets (QCS). Quadtree-based signaling strategy is used at the end of QNLK for adaptive filtering on/off control. Experimental results of QNLK show that the proposed method achieves significant objective coding gain and visual quality improvement, compared with both H.264/AVC high profile and HEVC.