Efficient intra- and inter-mode selection algorithms for H.264/ AVC

Intra-frame mode selection and inter-frame mode selection are new features introduced in the H.264 standard. Intra-frame mode selection dramatically reduces spatial redundancy in I-frames, while inter-frame mode selection significantly affects the output quality of P-/B-frames by selecting an optimal block size with motion vector(s) or a mode for each macroblock. Unfortunately, this feature requires a myriad amount of encoding time especially when a brute force full-search method is utilised. In this paper, we propose fast mode selection algorithms tailored for both intra-frames and inter-frames. The success of the intra-frame algorithm is achieved by reducing the computational complexity of the Lagrangian rate-distortion optimisation evaluation. Two proposed fast inter-frame mode algorithms incorporate several robust and reliable predictive factors, including intrinsic complexity of the macroblock, mode knowledge from the previous frame(s), temporal similarity detection and the detection of different moving features within a macroblock. This information is used to reduce the number of search operations. Extensive simulations on different classes of test sequences demonstrate a speed up in encoding time of up to 86% compared with the H.264 benchmark. This is achieved without any significant degradation in picture quality and compression ratio.