Flow distribution-aware load balancing for the datacenter

In this paper, we present a Flow Distribution-Aware Load Balancing (FDALB) mechanism to reduce flow completion times and achieve high scalability. In FDALB, flows are split into short flows and long flows according to a threshold. The traffic of short flows and long flows are balanced by distributed and centralized algorithms respectively. To adapt to traffic dynamics, we propose a simple yet effective scheme adaptively adjusting the splitting threshold. To reduce the overheads of classifying flows, FDALB leverages end-hosts to tag long flows, which requires no changes in networking hardware. To further reduce the overheads of handling flows, we proposed a new centralized algorithm without requiring flow rates information. Using realistic datacenter workloads, we show that FDALB reduces the average FCT of flows by up to 47% over ECMP while achieves higher scalability than the state-of-art load balancing mechanism Mahout.