Sources of uncertainty in model predictions: lessons learned from the IAEA Forest and Fruit Working Group model intercomparisons

The International Atomic Energy Agency (IAEA), through the BIOMASS program, has provided a unique international forum for assessing the relative contribution of different sources of uncertainty associated with environmental modeling. The methodology and guidance for dealing with parameter uncertainty have been fairly well developed and quantitative tools such as Monte-Carlo modeling are often recommended. The issue of model uncertainty is still rarely addressed in practical applications and the use of several alternative models to derive a range of model outputs (similar to what was done in IAEA model intercomparisons) is one of a few available techniques. This paper addresses the often overlooked issue of what we call 'modeler uncertainty,' i.e., differences in problem formulation, model implementation and parameter selection originating from subjective interpretation of the problem at hand. This study uses results from the Fruit and Forest Working Groups created under the BIOMASS program (BIOsphere Modeling and ASSessment). The greatest uncertainty was found to result from modelers' interpretation of scenarios and approximations made by modelers. In scenarios that were unclear for modelers, the initial differences in model predictions were as high as seven orders of magnitude. Only after several meetings and discussions about specific assumptions did the differences in predictions by various models merge. Our study shows that the parameter uncertainty (as evaluated by a probabilistic Monte-Carlo assessment) may have contributed over one order of magnitude to the overall modeling uncertainty. The final model predictions ranged between one and three orders of magnitude, depending on the specific scenario. This study illustrates the importance of problem formulation and implementation of an analytic-deliberative process in fate and transport modeling and risk characterization.