Two-level parallelization of a fluid mechanics algorithm exploiting hardware heterogeneity

•Heterogeneous computing is investigated for a typical CFD application.•MPI parallelizes the code, OpenMP and OpenACC accelerate it for multicores and GPGPUs.•Traditional load balancing models do not suffice for such heterogeneous systems.•A load balancing model for distributed heterogeneous systems is devised and evaluated.•The performance gains resulting from heterogeneous computing are close to optimal.

The prospect of wildly heterogeneous computer systems has led to a renewed discussion of programming approaches in high-performance computing, of which computational fluid dynamics is a major field. The challenge consists in harvesting the performance of all available hardware components while retaining good programmability. In particular the use of graphic cards is an important trend. This is addressed in the present paper by devising a hybrid programming model to create a heterogeneous data-parallel computation with a single source code. The concept is demonstrated for a one-dimensional spectral-element discretization of a fluid dynamics problem. To exploit the additional hardware available when coupling GPGPU-accelerated processes with excess CPU cores, a straight-forward load balancing model for such heterogeneous environments is developed. The paper presents a large number of run time measurements and demonstrates that the achieved performance gains are close to optimal. This provides valuable information for the implementation of fluid dynamics codes on modern heterogeneous hardware.