Energy-efficient multigrid smoothers and grid transfer operators on multi-core and GPU clusters

We investigate time and energy to solution for the CPU- and GPU-based execution of the compute intensive smoother and grid transfer operators in a geometric multigrid linear solver. We use a hybrid parallel implementation for both shared and distributed memory multi-core host systems comprising CUDA-capable devices. Our numerical experiments are designed to assess the effect of combining an MPI-parallel multigrid framework with OpenMP host threads or CUDA accelerators instead of MPI-only CPU computations for various parallel setups. We present runtime and energy measurements from a quad-CPU test system equipped with two GPUs. We find that using an accelerated asynchronous smoother can yield substantial savings of time and energy to solution over using a host-only Jacobi smoother in small and medium sized host systems with one or two multi-core CPUs. The acceleration of the grid transfer operators also yields a benefit, yet smaller than the benefit from the smoother. For large host systems a hybrid MPI-OpenMP parallelization turns out to be most beneficial with respect to energy consumption, although it is not the fastest option.