Feasible enhancements to congestion control in InfiniBand-based networks

The interconnection network architecture is crucial for High-Performance Computing (HPC) clusters, since it must meet the increasing computing demands of applications. Current trends in the design of these networks are based on increasing link speed, while reducing latency and number of components in order to lower the cost. The InfiniBand Architecture (IBA) is an example of a powerful interconnect technology, delivering huge amounts of information in few microseconds. The IBA-based hardware is able to deliver EDR and HDR speed (i.e. 100 and 200 Gb/s, respectively). Unfortunately, congestion situations and their derived problems (i.e. Head-of-Line blocking and buffer hogging), are a serious threat for the performance of both the interconnection network and the entire HPC cluster. In this paper, we propose a new approach to provide IBA-based networks with techniques for reducing the congestion problems. We propose Flow2SL-ITh, a technique that combines a static queuing scheme (SQS) with the closed-loop congestion control mechanism included in IBA-based hardware (a.k.a. injection throttling, ITh). Flow2SL-ITh separates traffic flows storing them in different virtual lanes (VLs), in order to reduce HoL blocking, while the injection rate of congested flows is throttled. Meanwhile congested traffic vanishes, there is no buffer sharing among traffic flows stored in different VLs, which reduces congestion negative effects. We have implemented Flow2SL-ITh in OpenSM, the open-source implementation of the IBA subnet manager (SM). Experimental results obtained by running simulations and real workloads in a small IBA cluster show that Flow2SL-ITh outperforms existing techniques by up to 44%, under some traffic scenarios.