Computation–communication overlap and parameter auto-tuning for scalable parallel 3-D FFT

• We design a new method of parallel 3-D FFT based on 2-D decomposition of an input 3-D array.
• We optimize the performance through computation–communication overlap and parameter auto-tuning.
• Experimental results from two supercomputers confirm that our method is faster than three existing libraries.

Parallel 3-D FFT is widely used in scientific applications, therefore it is important to achieve high performance on large-scale systems with many thousands of computing cores. This paper describes a new method for scalable high-performance parallel 3-D FFT. We use a 2-D decomposition of 3-D arrays to increase scaling to a large number of cores. In order to achieve high performance, we use non-blocking MPI all-to-all operations and exploit computation-communication overlap. We also auto-tune our 3-D FFT code efficiently in a large parameter space and cope with the complex trade-off in optimizing our code in various system environments. According to experimental results from two systems, our method computes parallel 3-D FFT significantly faster than three existing libraries, and scales well to at least 32,768 compute cores.