A fast intra prediction mode decision using DCT and quantization for H.264/AVC

Intra coding is used for reducing the spatial redundancy in video coding. H.264/AVC supports block-based predictions for Intra coding. These Intra predictions also have several directional prediction modes, which significantly improve the Intra coding efficiency, but increase the encoding complexity. In this paper, two fast Intra prediction mode decision algorithms are proposed for H.264/AVC Intra coding. The proposed algorithms select one representative prediction mode or a few representative prediction modes among all intra predictions using the existing DCT and quantization schemes of H.264/AVC, and only the representative modes are used for the Rate Distortion Optimization (RDO) process. The experimental results show that the proposed algorithm 1 and 2 achieve encoding time savings of 53% and 63%, respectively, with negligible loss of Peak Signal-to-Noise Ratio (PSNR).

► H.264/AVC supports block-based predictions for Intra coding to reduce the spatial redundancy.
► In this study, two fast Intra prediction mode decision algorithms are proposed for H.264/AVC Intra coding.
► Quantization schemes of H.264/AVC, and only the representative modes are used for RDO (Rate Distortion Optimization) process.