HEVC intra mode selection based on Rate Distortion (RD) cost and Sum of Absolute Difference (SAD)

• The relationship between SAD and RD-cost is investigated for intra mode selection.
• The ‘rate’ and ‘distortion’ requirement for a given SAD value varies with the QP.
• The model is derived that is based on the rate and distortion requirement for SAD.
• The overall proposed algorithm efficiently short list the intra mode(s) for RDO.
• The RMD module of HEVC is outperformed in terms of BD-Rate and time saving.

High Efficiency Video Coding (HEVC) encoder provides higher compression efficiency by offering 35 intra modes. However, the encoding complexity is increased due to more modes are involved in the decision process. Therefore, it is desired to build a fast intra prediction algorithm that is practical for real time application. In this paper, a quadratic approach for reducing intra coding complexity is proposed. Firstly, the relationship between the RD-cost and the SAD is investigated. Secondly, a model is proposed to estimate the RD-cost of all 35 intra modes using the quadratic relation, thus avoiding the computation of entropy coding, Hadamard cost, distortion, and transform. Experimental results demonstrate that the average time saving of the proposed approach is 31–38%, while the BD-Bit Rate increment is only 0.62–1.37%, respectively.