A machine-learning approach to multi-robot coordination

This paper presents a machine-learning approach to the multi-robot coordination problem in an unknown dynamic environment. A multi-robot object transportation task is employed as the platform to assess and validate this approach. Specifically, a flexible two-layer multi-agent architecture is developed to implement multi-robot coordination. In this architecture, four software agents form a high-level coordination subsystem while two heterogeneous robots constitute the low-level control subsystem. Two types of machine learning—reinforcement learning (RL) and genetic algorithms (GAs)—are integrated to make decisions when the robots cooperatively transport an object to a goal location while avoiding obstacles. A probabilistic arbitrator is used to determine the winning output between the RL and GA algorithms. In particular, a modified RL algorithm called the sequential Q-learning algorithm is developed to deal with the issues of behavior conflict that arise in multi-robot cooperative transportation tasks. The learning-based high-level coordination subsystem sends commands to the low-level control subsystem, which is implemented with a hybrid force/position control scheme. Simulation and experimental results are presented to demonstrate the effectiveness and adaptivity of the developed approach.