A framework for fast mode decision in the H264 video coding standard

We propose a novel framework for fast mode decision in the simple and main profiles of the H264 video coding standard. Our framework consists of a specific combination of algorithms, each achieving computational savings while retaining rate distortion (RD) performance very similar to the standard. In particular, we utilise a set of skip mode conditions for P and B slices, two heuristics that reduce the cardinality of the inter mode set to be examined, inter/intra mode prediction and the monotonicity property of the rate distortion cost functions. We achieve content dependent savings in run times between 5.8 and 90.1% as compared to H264. Compared to other work that was used as input to the standard, our scheme is faster by 9–23% for very similar RD performance. The proposed framework can be used wholly or partially for computational speed-ups, it is independent of the motion search method used and is applicable in both the rate controlled and nonrate controlled cases.