A comparison of four wheat models with respect to robustness and transparency: Simulation in a temperate, sub-humid environment

There is debate over determining the appropriate model complexity to simulate crop development, growth, and yield. An approach that is sometimes suggested is to compare the performance of models using common datasets for ability to reproduce specific sets of observations. However, this narrow-focused approach overlooks the critical heuristic aspects in using models to explore and understand the behavior of cropping systems at the process level. We argue that the key criteria of model evaluation are both transparency and overall robustness. While model robustness (often mislabeled as “validation”) is sometimes presented at some level, model transparency has normally been ignored in model comparison studies. The objective of this paper is to examine the transparency and robustness of four wheat (Triticum aestivum L.) models that are markedly different in detail: CropSyst and SSM as simpler models and APSIM and DSSAT as more complex models. Data for development, growth and yield of the crop were collected from a wide range of environmental and growth conditions in the Grogan region of Iran. Models parameterization was done according to the guidelines for each model and then model testing and comparison were performed using different datasets. The two simpler models were found to be more robust than the complex models; across all the evaluated crop variables, the coefficient of variation in yield prediction was lower for SSM (8.2%) and CropSyst (14.3%) than APSIM (15.0%) and DSSAT (18.5%). Transparency of the models was mainly gauged by the number of input parameters needed by the models. Simulations using APSIM (292 parameters) and DSSAT (211 parameters) required the definition of about fourfold more parameters than CropSyst (50 parameters) and SSM (55 parameters). The simulation results showed no significant relationship between model performance and parameter number; the lack of transparency sacrificed in complexity was not rewarded by increased robustness in the output.