Fast H.264 Intra-prediction mode selection using joint spatial and transform domain features

A fast H.264 Intra-prediction mode selection scheme is proposed in this work. The objective is to reduce the encoder complexity without significant rate–distortion performance degradation. The proposed method uses spatial and transform domain features of the target block jointly to filter out the majority of candidate modes. This is justified by examining the posterior error probability and the average rate–distortion loss. For the final mode selection, either the feature-based or the RDO (rate–distortion optimization)-based method is applied to 2–3 candidate modes. It is demonstrated by experimental results that the proposed scheme demands only 7–10% of the complexity of the RDO (rate–distortion optimized) mode decision scheme with little quality degradation.