Solving Network Design Problem with Dynamic Network Loading Profiles Using Modified Reinforcement Learning Method

This study aims to solve dynamic user Equilibrium Network Design Problem (ENDP) with dynamic network loading profiles using modified Reinforcement Learning (RL) approach. The bi-level programming technique is used to solve the problem. At the lower level of the problem, the dynamic User Equilibrium (UE) link flows are obtained by simulation based Dynamic Traffic Assignment (DTA) model with DynusT and signal timings are obtained at the upper level by modified RL method. The system Performance Index (PI) is defined as the sum of a weighted linear combination of delay and number of stops per unit time for all traffic streams, which is evaluated by the traffic model of TRANSYT-7F. Q-learning, a model-free approach, is one of the RL methods. The modified RL method is actually based on Q-learning. By integrating the modified RL method, traffic assignment and traffic control, the modified REinforcement Learning TRANSYT-7F DynusT (RELTRAD) model is proposed to solve the dynamic ENDP. The objective function of the proposed RELTRAD is total network PI. The model is tested on the medium sized Allsop and Charlesworth's network. Two scenarios, related to various dynamic network loading profiles, are proposed for numerical application. Encouraging results are obtained. Results showed that the RELTRAD model effectively optimizes the signal timings and values of the network PI. The RELTRAD model improves to the network PI from the initial value to the final value as 65% and 67% for loading profile 1 and 2, respectively.