Model validation and predictive capability for the thermal challenge problem

We address the thermal problem posed at the Sandia Validation Challenge Workshop. Unlike traditional approaches that confound calibration with validation and prediction, our approach strictly distinguishes these activities, and produces a quantitative measure of model-form uncertainty in the face of available data. We introduce a general validation metric that can be used to characterize the disagreement between the quantitative predictions from a model and relevant empirical data when either or both predictions and data are expressed as probability distributions. By considering entire distributions, this approach generalizes traditional approaches to validation that focus only on the mean behaviors of predictions and observations. The proposed metric has several desirable properties that should make it practically useful in engineering, including objectiveness and robustness, retaining the units of the data themselves, and generalizing the deterministic difference. The metric can be used to assess the overall performance of a model against all the experimental observations in the validation domain and it can be extrapolated to express predictive capability of the model under conditions for which direct experimental observations are not available. We apply the metric and the scheme for characterizing predictive capability to the thermal problem.