Reward functions for learning to control in air traffic flow management

• Reinforcement learning and its reward structure are investigated for ATFM decision making.
• For GHP, the reward function is developed in considering the safety factors and fairness impact.
• For AHP, another reward function is developed to consider the safety factors.
• Real case studies show the effectiveness and efficiency of the developed method.

Air Traffic Flow Management (ATFM) is a complex decision-making process with multiple stakeholders involved. In this decision loop, a Multi-agent system is developed for both simulation and daily operations to support human decisions. Considering human factors in ATFM, the method of Reinforcement Learning (RL) is suitable in the acquirement of the knowledge and experience of the controllers to assist them in the next control activities. The paper presents the recent development of reinforcement learning and its reward structure for ATFM decision making. Two types of reward functions are proposed for agent-based RL in the application of air traffic management: (1) Reward function considering safety separation and fairness impact among different commercial entities in Ground Holding Problem (GHP) and (2) Reward function considering safety separation in Air Holding Problem (AHP). Real case studies in Brazil are described to show the effectiveness and efficiency of the developed reward functions in the controller decision process of ATFM.