KNEM: A generic and scalable kernel-assisted intra-node MPI communication framework

The multiplication of cores in today’s architectures raises the importance of intra-node communication in modern clusters and their impact on the overall parallel application performance. Although several proposals focused on this issue in the past, there is still a need for a portable and hardware-independent solution that addresses the requirements of both point-to-point and collective MPIoperations inside shared-memory computing nodes.This paper presents the KNEM module for the Linux kernel that provides MPI implementations with a flexible and scalable interface for performing kernel-assisted single-copy data transfers between local processes. It enables high-performance communication within most existing MPI implementations and brings significant application performance improvements thanks to more efficient point-to-point and collective operations.

► In-depth analysis of current intra-node communication state of the art.
► Extensive analysis of MPI requirements and of the KNEM interface.
► KNEM offers portable, flexible and scalable intra-node MPI communication.
► KNEM benefits point-to-point and collective operations.
► KNEM is already used in most existing MPI implementations.