Adaptive bit allocation scheme for rate control in high efficiency video coding with initial quantization parameter determination

•The relationship between coding bit rate and texture complexity is represented by a linear function.•We develop a more accurate texture complexity measurement model.•We develop an adaptive bit allocation scheme for rate control in HEVC.•The initial quantization parameter depends on the bits per pixel and the video contents.•An adaptive quantization parameter clip range determination scheme to achieve the visual quality smoothness and prevent the encoder buffer from underflow and overflow.

One of the challenges in rate control (RC) lies in how to efficiently determine a target bit rate that will be used for the quantization parameter (Qp) calculation process during video coding. In this paper, we investigate the issues over the existing bit allocation algorithms for the RC process in High Efficiency Video Coding (HEVC) and propose an complexity-based bit allocation scheme to improve the encoding performance. First, we model the relationship between encoding bit rate and texture complexity by a linear rate function. Second, compared with traditional complexity estimation methods, a more accurate model is proposed to measure the texture complexity considering the spatial–temporal correlations. Third, based on the proposed rate function and texture complexity measurement model, we develop an adaptive bit allocation scheme for RC in HEVC. At the same time, depending on the encoder buffer status, an adaptive Qp clip range determination algorithm is also developed to achieve the encoding quality smoothness while keeping the bit rate fluctuation at an acceptable level. Then, we exploit to determine the initial Qp efficiently and adaptively according to video contents. Experimental results demonstrate that the proposed RC algorithm can achieve better rate-distortion (R–D) and rate-control performance than that of the state-of-the-art RC scheme implemented in the HEVC reference software HM11.0.