An analytic model for quantized discrete transform error

Almost every method currently used in the technology of multimedia compression is based on and operates by some type of quantized discrete transform. Due to quantization, there is a difference between the data before and after such a transform usage that can be calculated simply as the (R)MSE (root-mean-square-error) of one to another. In this work, we model any such (R)MSE estimate as a function of the transform itself, which is assumed to be given by its square matrix, of the quantization given by an array of divisors and, finally, of the source data statistics, which are given by the data means and covariances. The model has been designed for one- and two-dimensional transforms and can be applied to any of them, including DCT (discrete cosine transform), DWT (discrete Walsh transform), DFT (discrete Fourier transform), and wavelets. Trial calculations performed for the DCT II, DCT I, biorthogonal DCT I, Haar and Walsh transforms show that the results are very close to those predicted by the model.