Utility optimal scheduling in processing networks

We consider the problem of utility optimal scheduling in general processing networks with random arrivals and network conditions. These are generalizations of traditional data networks where commodities in one or more queues can be combined to produce new commodities that are delivered to other parts of the network, and can be used to model problems such as in-network data fusion, stream processing, MapReduce scheduling, and grid computing. Scheduling actions are complicated by the underflow problem that arises when some queues with required components go empty. In this paper, we develop a novel methodology for constructing and analyzing online   algorithms for such processing networks. Specifically, we develop the Perturbed Max-Weight algorithm (PMW) to achieve optimal utility. The idea of PMW is to perturb the weights used by the usual Max-Weight algorithm to “push” queue levels toward non-zero values (avoiding underflows). We then show, using a novel combination of Lyapunov drift analysis and duality theory, that when the perturbations are carefully chosen, PMW is able to achieve a utility that is within O(1/V)O(1/V) of the optimal value for any V≥1V≥1, while ensuring an average network backlog of O(V)O(V). The methodology developed here is very general and can also be applied to other problems that involve such underflow constraints.