Methodology to evaluate the performance of simulation models for alternative compiler and operating system configurations

Simulation modelers increasingly require greater flexibility for model implementation on diverse operating systems, and they demand high computational speed for efficient iterative simulations. Additionally, model users may differ in preference for proprietary versus open-source software environments. These issues necessitate the development of strategies to maximize model compatibility across operating systems, to ensure numerically accurate simulations for alternative compiler selections, and to understand how these choices affect computational speed. We developed an approach to evaluate model performance using diverse Fortran compilers on multiple computer operating systems. A single desktop computer with five identical hard drives was designed to permit meaningful comparisons between five operating systems while minimizing differences in hardware configuration. Three Fortran compilers and relevant software development tools were installed on each operating system. Both proprietary and open-source versions of compilers and operating systems were used. Compatibility and performance issues among compiler and operating system combinations were assessed for an example case: the Cropping System Model (CSM) as implemented in version 4.5 of the Decision Support System for Agrotechnology Transfer (DSSAT). A simulation study that included 773 simulations and assessed the full suite of crop growth modules within DSSAT-CSM was conducted for each compiler and operating system configuration. For a given simulation, results were identical for anthesis date (ADAT), maturity date (MDAT), and maximum leaf area index (LAIX) regardless of the compiler or operating system used. Over 94% of the simulations were identical for canopy weight at maturity (CWAM) and cumulative evapotranspiration at maturity (ETCM). Differences in CWAM were predominantly less than 2 kg ha−1 and were likely the result of differences in floating point handling among compilers. Larger CWAM discrepancies highlighted areas for improvement of the model code. Model implementations with the Intel Fortran compiler on the Linux Ubuntu operating system provided the fastest simulations, which averaged 9.0 simulations s−1. Evaluating simulation models for alternative compiler and operating system configurations is invaluable for understanding model performance constraints and for improving model robustness, portability, usefulness, and flexibility.

► We compare simulation model performance using various compilers and operating systems.
► We develop a methodology to compare the numerical accuracy and speed of simulation models.
► The methodology can improve the robustness and portability of simulation model code.
► The methodology is useful for extending Windows-based simulation models to Linux.
► The methodology identifies model implementation options that increase computational speed.