Fast sum of absolute transformed difference based 4×4 intra-mode decision of H.264/AVC video coding standard

One of the new features in the H.264/AVC encoder is the use of Lagrangian rate-distortion optimization (RDO) method during mode decision at the macroblock level. The RDO technique has been employed in H.264/AVC for intra-prediction mode selection to achieve better coding efficiency. But the computational complexity of mode decision algorithm is extremely high. To reduce the complexity of mode decision, we propose an efficient and fast 4×4 intra-prediction mode selection scheme. The proposed method reduces the candidate of the prediction modes based on the correlation between neighboring blocks and the sum of absolute transformed difference (SATD) between the original block and the intra-predicted block. Firstly, the rank of each mode is obtained based on the SATD value. Then, the candidate modes are further reduced by using the combination of rank and most probable mode. The proposed method reduces the number of candidate mode to either one or two. Simulation results demonstrate that the proposed mode decision method reduces about 91% of mode decision time and 70% of total encoding time of intra-coding with ignorable degradation of coding performance.