A performance evaluation of the National Air Quality Forecast Capability for the summer of 2007

This paper provides a performance evaluation of the real-time, CONUS-scale National Air Quality Forecast Capability (NAQFC) that supported, in part, its transition into operational status. This evaluation focuses primarily on discrete forecasts for the maximum 8-h O3 concentrations covering the 4-month period, June through September, 2007, using measurements obtained from EPA's AIRNow network. Results indicate that the 2007 NAQFC performed as well or better than previous configurations, despite the expansion of the forecast domain into the western half of the nation that is dominated by complex terrain. The mean, domain-wide, season-long correlation was 0.70. When examined over time, the domain-wide correlations exhibit a fairly consistent nature, with values exceeding 0.60 (0.70) over 90% (55%) of the days. The NAQFC systematically over-predicted the 8-h O3 concentrations, continuing a trend established by earlier NAQFC configurations, though to a lesser degree. The summer-long mean forecast value of 53.2 ppb was 4.2 ppb higher than the observed value, resulting in a domain-wide Normalized Mean Bias (NMB) of 8.7%. Most of the over-prediction is associated with observed concentrations less than 50 ppb. In fact the model tends to under-predict when concentrations exceed 70 ppb. As with the bias, the error associated with the latest configuration was also lower. The summer-long Root Mean Square Error of 13.0 ppb (Normalized Mean Error (NME) = 20.4%) represented marked improvements over earlier forecasts. Examination of the spatial distribution of both the NMB and NME reveals that the NAQFC was generally within 25% for the NME and 25% for the NMB over a majority of the domain. Several areas of poorer performance, where the NMB and NME often exceed 25% and in some cases 50%, were noted. These areas include southern California, where the NAQFC tended to under-predict concentrations (especially on weekends) and the southeast Atlantic and Gulf coasts regions, where the model over-predicted. Subsequent analysis revealed that the incorrect temporal allocation of precursor emissions was likely the source of the under-prediction in southern California, while inaccurate simulation of PBL heights likely contributed to the over-prediction in the coastal regions.