Uncertainty in air quality model evaluation for particulate matter due to spatial variations in pollutant concentrations

Air quality model performance is usually evaluated by examining the relative agreement between volume-averaged simulations and point measurements, as volume-averaged measurements are seldom available. Because the two values have different spatial scales, accurate model evaluation is complicated by this mismatch in areas when the pollutant gradient is large. Uncertainty in the air quality model evaluation from the spatial variability of PM2.5 is quantitatively examined, and how much of model error might be explained by such variability is calculated. Added uncertainty of model performance is analyzed by comparing performance metrics between simulated concentrations and observations at one station between simulated levels and interpolated fields from observations. Normalized differences of the performance metrics (e.g., mean fractional error, MFE) calculated in these two ways indicate the uncertainty of the model performance due to spatial variation. Normalized difference of MFE for PM2.5 mass is approximately 17% in July 2001 and 15% in January 2002. To decrease the uncertainty, it has been suggested that observations be used only from spatially representative stations. When model performance is calculated with data from spatially representative stations, uncertainty decreased, and overall model performance improves. For example, MFE is seen to decrease up to 14% for PM2.5 mass and species concentrations, suggesting that up to 14% of MFE can be explained by the spatial variability of PM2.5. These results indicate that comparison between observed and simulated concentrations should not be used alone to assess performance of air quality models. Also, spatial variability should be considered in setting model performance goals.