Implementing discrete element method for large-scale simulation of particles on multiple GPUs

Recently, the use of graphics processing units (GPUs) has become popular in scientific computations due to their low cost, impressive floating-point capabilities, high memory bandwidth, and low electrical power requirements. Discrete element method (DEM), closely related to molecular dynamics (MD), is a widely-accepted method for simulating particle systems, such as powders, bubbles, grains, etc. In this work, we present an efficient implementation of DEM on multiple GPUs. Linked-cell list method is used to accelerate recognition of contacting particles and is beneficial for communications among multiple GPUs. Moreover, asynchronous communication is used to reduce the time costs of communications. Physical validation is performed and parallel efficiency is evaluated, showing that our implementation has high computational efficiency and good scalability. The simulation of a large-scale dense particle system (up to 128,000,000) is achieved with a good parallel efficiency. The speed-up ratio in 16 GPUs vs 128 CPU cores is 10.39.