Advances in patch-based adaptive mesh refinement scalability

• We developed two key SAMR regridding components that scaled individually and integrated scalably.
• The cascade partitioner took 10% of the regrid time and yielded loads within 10% of ideal.
• The tile clustering step took about 2% of regrid time and reduced cluster counts by a factor of 38.
• Our benchmarks, set up to be challenging for dynamic adaptivity, scaled to 2M MPI tasks.
• Smooth, well-behaved timer trends indicate higher scaling is possible.

Patch-based structured adaptive mesh refinement (SAMR) is widely used for high-resolution simulations. Combined with modern supercomputers, it could provide simulations of unprecedented size and resolution. A persistent challenge for this combination has been managing dynamically adaptive meshes on more and more MPI tasks. The distributed mesh management scheme in SAMRAI has made some progress SAMR scalability, but early algorithms still had trouble scaling past the regime of 105105 MPI tasks. This work provides two critical SAMR regridding algorithms, which are integrated into that scheme to ensure efficiency of the whole. The clustering algorithm is an extension of the tile-clustering approach, making it more flexible and efficient in both clustering and parallelism. The partitioner is a new algorithm designed to prevent the network congestion experienced by its predecessor. We evaluated performance using weak- and strong-scaling benchmarks designed to be difficult for dynamic adaptivity. Results show good scaling on up to 1.5M cores and 2M MPI tasks. Detailed timing diagnostics suggest scaling would continue well past that.