Design a deblocking filter with three separate modes in DCT-based coding

The reconstructed images from highly compressed data have noticeable image degradations, such as blocking artifacts near the block boundaries. Post-processing appears to be the most feasible solution because it does not require any existing standards to be changed. Markedly reducing blocking effects can increase compression ratios for a particular image quality or improve the quality of equally compressed images. In this work, a novel deblocking algorithm is proposed based on three filtering modes in terms of the activity across block boundaries. By properly considering the masking effect of the HVS (Human Visual System), an adaptive filtering decision is integrated into the deblocking process. According to three different deblocking modes appropriate for local regions with different characteristics, the perceptual and objective quality are improved without excessive smoothing the image details or insufficiently reducing the strong blocking effect on a flat region. According to the simulation results, the proposed method outperforms other deblocking algorithms in respect to PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural SIMilarity).