Performance criteria to evaluate air quality modeling applications

A set of statistical indicators fit for air quality model evaluation is selected based on experience and literature: The Root Mean Square Error (RMSE), the bias, the Standard Deviation (SD) and the correlation coefficient (R). Among these the RMSE is proposed as the key one for the description of the model skill. Model Performance Criteria (MPC) to investigate whether model results are 'good enough' for a given application are calculated based on the observation uncertainty (U). The basic concept is to allow for model results a similar margin of tolerance (in terms of uncertainty) as for observations. U is pollutant, concentration level and station dependent, therefore the proposed MPC are normalized by U. Some composite diagrams are adapted or introduced to visualize model performance in terms of the proposed MPC and are illustrated in a real modeling application. The Target diagram, used to visualize the RMSE, is adapted with a new normalization on its axis, while complementary diagrams are proposed. In this first application the dependence of U on concentrations level is ignored, and an assumption on the pollutant dependent relative error is made. The advantages of this new approach are finally described.

► New method to evaluate air quality models based on observation uncertainty. ► The same margin of tolerance is allowed for models as for observations. ► New criteria and diagrams to visualize fulfillment zones are described. ► Various dependencies (pollutant, concentrations level…) can be considered. ► Example diagrams and criteria values from measurements are provided.