Dynamic evaluation of CMAQ part II: Evaluation of relative response factor metrics for ozone attainment demonstrations

The U.S. Environmental Protection Agency provides guidelines on the use of air quality models for projecting whether an emission reduction strategy will lead to future pollutant levels that are at or below the National Ambient Air Quality Standards (NAAQS). The EPA's guidance document for ozone attainment demonstrations recommends an attainment test for the 8-h ozone NAAQS based on using the ratio of output from “future” and “base” model simulations through the calculation of location-specific Relative Response Factors (RRF). The 2007 guidance document as well as other related studies have recommended the use of retrospective evaluation studies in order to evaluate the ability of an air quality model to represent a change in air quality (dynamic evaluation) rather than relying solely on operational evaluation of model predictions under base line conditions. Here simulations from the Community Multiscale Air Quality (CMAQ) modeling system were conducted for 2002 and 2005, a time period characterized by significant emissions reductions associated with the EPA's Nitrogen Oxides State Implementation Plan Call (NOx SIP Call) as well as mobile sources. These simulations were used to evaluate the performance of different forms of the RRF metric for projecting 2002 to 2005 against 2005 observed ozone levels. The evaluation study showed that the current form of the RRF calculation is generally well designed for predicting the future 8-hr ozone “design value” metric used for determining attainment. Specifically, the methodology of using air quality model simulations in a relative sense provided better estimates of future ozone design values than using the modeled future year simulation alone. Alternative forms of the RRF metric were found to be very similar to the current methodology in terms of evaluation metrics. However, alternative RRF metrics were sensitive to the number of days used in the calculation of the RRF. Approaches which used more days in the RRF calculation (relative to the 2007 guidance approach) had slightly higher bias and error in predicting 2005 design values compared to approaches using only a subset of high ozone days.