MDID: A multiply distorted image database for image quality assessment

•The images are multiply distorted with random types and levels of distortions.•We propose a subjective evaluation scheme based on pair comparison sorting.•The initial parameters are estimated for regression through mathematical derivation.•Experiments validate the reasonability and challenges of this database.

In this paper, we present a new database, the multiply distorted image database (MDID), to evaluate image quality assessment (IQA) metrics on multiply distorted images. The database contains 20 reference images and 1600 distorted images. The latter images are obtained by contamination of the former with multiple distortions of random types and levels, so multiple types of distortions appear in each distorted image. Pair comparison sorting (PCS) is used as a new subjective rating method to evaluate image quality. This method allows subjects to make equal decisions on images whose difference in quality cannot be easily evaluated visually. A total of 192 subjects participated in the subjective rating, in which mean opinion scores and standard deviations were obtained. In IQA research, subjective scores and algorithm predictions are generally related by a nonlinear regression. We further propose a method to initialize the parameters of the nonlinear regression. The experiments of IQA metrics conducted on MDID validate that this database is advisable and challenging.