Replicating complex agent based models, a formidable task

- Cross language replication of ABMs may introduce variability in replicate models.
- A complex ABM was replicated in two different development platforms and languages.
- Data generated by the models differed in magnitude, trend and conclusions drawn.
- Crucial to consider cross language/platform when assessing repeatability.

Promoting replication of models is unarguably a positive step for agent based modelling, as replication promotes rigorous testing. Model replication remains rare, yet is vital to assessing the repeatability of existing agent based models. Notably, more work is needed to assess cross platform and language replication, which represent potential sources of variability between model results. An existing, complex agent based model was replicated using two widely used platforms (NetLogo and Repast). When results generated by the models were compared, the findings differed not only in magnitude but the trends produced by the data, resulting in different conclusions being drawn from each set of model predictions. The variation between the models is believed to be a result of the complexity of encoding a substantial theoretical model in particular programming languages. This highlights the express need to document replication of existing models in order to fully understand the potential limitations to replication.