A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework

• Propose a coordinated multi-agent reinforcement learning framework for signal control.
• Propose the Junction Tree Algorithm to obtain best joint actions of coordinated intersections.
• Various scenarios are constructed and multiple tests are conducted in the case study.
• Assess the delay, number of stops, and environmental impacts of the proposed controller.
• Results show that the proposed algorithm outperforms other learning based algorithms.

This study develops a novel reinforcement learning algorithm for the challenging coordinated signal control problem. Traffic signals are modeled as intelligent agents interacting with the stochastic traffic environment. The model is built on the framework of coordinated reinforcement learning. The Junction Tree Algorithm (JTA) based reinforcement learning is proposed to obtain an exact inference of the best joint actions for all the coordinated intersections. The algorithm is implemented and tested with a network containing 18 signalized intersections in VISSIM. Results show that the JTA based algorithm outperforms independent learning (Q-learning), real-time adaptive learning, and fixed timing plans in terms of average delay, number of stops, and vehicular emissions at the network level.