Monte Carlo simulation on heterogeneous distributed systems: A computing framework with parallel merging and checkpointing strategies

This paper introduces an end-to-end framework for efficient computing and merging of Monte Carlo simulations on heterogeneous distributed systems. Simulations are parallelized using a dynamic load-balancing approach and multiple parallel mergers. Checkpointing is used to improve reliability and to enable incremental results merging from partial results. A model is proposed to analyze the behavior of the proposed framework and help tune its parameters. Experimental results obtained on a production grid infrastructure show that the model fits the real makespan with a relative error of maximum 10%, that using multiple parallel mergers reduces the makespan by 40% on average, that checkpointing enables the completion of very long simulations and that it can be used without penalizing the makespan.

► We propose a new framework for Monte Carlo simulations on heterogeneous DCIs.
► We propose a map-reduce approach with multiple parallel reducers and checkpointing.
► On average merging time is reduced with 40% when using multiple parallel mergers.
► We propose a model to explain the measures made in production.
► Experimental results fit the model with a relative error of less than 10%.