16 000–64 000 B pMTU experiments with simulation: The case for super jumbo frames at Supercomputing ’05

This paper presents a case study including the results and preliminary simulations for a series of Ethernet-based Xnet “super jumbo frame” (SJF) experiments conducted prior to and at Supercomputing ’05, for up-to-64 000 B path MTU. As far as we are aware, the first public supercomputing demonstration of a router passing 64 000 B frames was performed in our Xnet SJF booth. While Gigabit Ethernet based network architectures typically show reduced zero payload node latency performance relative to interconnect solutions (Myrinet, SP Switch and InfiniBand), some with higher path MTU equivalent, Ethernet continues to be the most widely deployed and may remain so for the forseeable future. Cumulative jumbo frame research spanning several years, combined with theoretical calculations and extrapolations from experimental data obtained during Supercomputing ’05 indicates the possible practical feasibility of SJF-based network mechanics as a potential means to realize practical long term performance goals for high throughput streaming. Some of the lessons and implications of the SJF approach are discussed in relation to the evolution of novel network architectures, particularly in relation to explicit path systems for the high performance computing community, pending deployment of 40 and/or 100 Gb Ethernet.