Optimal allocation of servers and processing time in a load balancing system

We consider the problem of allocating processing time in a multi-channel load balancing system by focusing on systems where processing times have distributions characterized by high variability. Our objective is to reduce congestion by routing jobs to servers based on their workload. Specifically, we arrange servers in two stations in series, and require that the load be balanced between the two stations. All arrivals join the first service center where they receive a maximum of T units of service. Arrivals with service requirements that exceed the value T join the second station where they receive their remaining service. For a variety of heavy tail service time distributions, characterized by high variability, analytical and numerical comparisons show that our scheme provides better system performance than the standard parallel multi-server model in the sense of reducing the mean delay per customer when the traffic intensity is not too low. In particular, we develop lower bounds on the traffic intensity and the service time coefficient of variation beyond which the balanced series system outperforms the parallel system.