Traffic flow optimization: A reinforcement learning approach

• We model a traffic flow optimization problem as a reinforcement learning problem.
• We show how speed limit policies can be obtained using Q-learning.
• Neural networks improve the performance of our policy learning algorithm.
• Resulting policies are able to significantly reduce traffic congestion.
• Our method takes traffic predictions into account and controls proactively.

Traffic congestion causes important problems such as delays, increased fuel consumption and additional pollution. In this paper we propose a new method to optimize traffic flow, based on reinforcement learning. We show that a traffic flow optimization problem can be formulated as a Markov Decision Process. We use Q-learning to learn policies dictating the maximum driving speed that is allowed on a highway, such that traffic congestion is reduced. An important difference between our work and existing approaches is that we take traffic predictions into account. A series of simulation experiments shows that the resulting policies significantly reduce traffic congestion under high traffic demand, and that inclusion of traffic predictions improves the quality of the resulting policies. Additionally, the policies are sufficiently robust to deal with inaccurate speed and density measurements.