Xen2MX: High-performance communication in virtualized environments

• We discuss current methods of network access in virtualization platforms.
• We identify key design choices for a VM-aware cluster interconnection protocol.
• We introduce Xen2MX, a high-performance interconnection protocol for virtualized environments.
• Xen2MX semantically enriches the guest-to-host communication.
• Xen2MX is able to saturate a 10 Gbps link without the necessity to use specialized hardware.

Cloud computing infrastructures provide vast processing power and host a diverse set of computing workloads, ranging from service-oriented deployments to high-performance computing (HPC) applications. As HPC applications scale to a large number of VMs, providing near-native network I/O performance to each peer VM is an important challenge. In this paper we present Xen2MX, a paravirtual interconnection framework over generic Ethernet, binary compatible with Myrinet/MX and wire compatible with MXoE. Xen2MX combines the zero-copy characteristics of Open-MX with Xen's memory sharing techniques. Experimental evaluation of our prototype implementation shows that Xen2MX is able to achieve nearly the same raw performance as Open-MX running in a non-virtualized environment. On the latency front, Xen2MX performs as close as 96% to the case where virtualization layers are not present. Regarding throughput, Xen2MX saturates a 10 Gbps link, achieving 1159 MB/s, compared to 1192 MB/s of the non-virtualized case. Scales efficiently with the number of VMs, saturating the link for even smaller messages when 40 single-core VMs put pressure on the network adapters.