Data movement optimizations for independent MPI I/O on the Blue Gene/Q

Scalable high-performance I/O is crucial for application performance on large-scale systems. With the growing complexity of the system interconnects, it has become important to consider the impact of network contention on I/O performance because the I/O messages traverse several hops in the interconnect before reaching the I/O nodes or the file system. In this work, we present a route-aware and load-aware algorithm to modify existing bridge node assignment in the Blue Gene/Q (BG/Q) supercomputer. We reduce the network contention and reduce the write time by an average of 60% over the default independent I/O and by 20% over collective I/O on up to 8192 nodes on the Mira BG/Q system. Our algorithm routes 1.4× fewer messages through the bridge nodes which connect to the I/O nodes on the BG/Q. Our algorithm also reduces the average distance of a compute node from a bridge node, and thus lessens the network load, and decreases I/O time.