A scalable method for DCLC problem using hierarchical MDP model

It is well known that the delay-constrained least-cost (DCLC) routing problem is NP-complete, hence various heuristic methods have been proposed for this problem. However, these heuristic methods have poor scalability as the network scale increases. In this paper we propose a new method based on the Markov Decision Process (MDP) theory and the hierarchical routing scheme to address the scalability issue of the DCLC routing problem. We construct a new two-level hierarchy MDP model and apply an infinite-horizon discounted cost model to the upper level for the end-to-end inter-domain link selection. Since the infinite-horizon discounted cost model is independent of network scale, the scalability problem is resolved. With the proposed model, we further give the algorithm of solving the optimal policy to obtain the DCLC routing. Simulation results show that the proposed method improves the scalability significantly.