Optimization of hybrid stochastic differential systems in communications networks

This paper addresses issues from applied stochastic analysis for modeling and solving control problems in communications networks. We consider the problem of optimal scheduling in a wireless system with time varying traffic. The system is handled by a single base station transmitting over time varying channels. This may be the case in practice for a hybrid TDMA–CDMA (Time Division Multiple Access–Code Division Multiple Access) system. Heavy traffic approximation for the physical system yields a problem of optimal control in a constrained hybrid stochastic differential system. Here constrained means bounded or reflected in the KK-dimensional positive orthant. In this work we establish a closed form solution for the nascent optimal control problem. The aim is to find a control which minimizes the expected total delay for the users. The control is constrained to satisfy some inequalities. Hence it seems natural that the cost function involve a penalty term for breaking these inequalities. We study this control problem by a dynamic programming approach and we are led to the resolution of a Hamilton–Jacobi–Bellman (HJB) equation in the finite dimensional space R+K. While optimizing it turns out that the optimum falls inside the class of feedback controls. Hence our method consists in finding a smooth solution of Bellman’s equation and consequently getting a unique solution for the closed loop. Here a separation of variables enables us to construct an explicit C2C2 solution of the HJB equation so that existence of a unique optimal control is proven. Further stochastic analysis of the hybrid stochastic differential system under the optimal control is provided: the Fokker–Planck equation for the distribution density of the state.