Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games

In this paper, a new optimal distributed synchronization control scheme for the consensus problem of heterogeneous multi-agent differential graphical games is developed by iterative adaptive dynamic programming (ADP). The main idea is to use iterative ADP technique to obtain the iterative control law which makes all the agents track a given dynamics and simultaneously makes the iterative value function reach the Nash equilibrium. In the developed heterogeneous multi-agent differential graphical games, the agent of each node is different from one another. The dynamics and performance index function for each node depend only on local neighborhood information. A cooperative policy iteration algorithm is presented to achieve the optimal distributed synchronization control law for the agent of each node, where the coupled Hamilton–Jacobi equations for optimal synchronization control of heterogeneous multi-agent differential games can be avoided. Convergence analysis is developed to show that the iterative value functions of heterogeneous multi-agent differential graphical games can converge to the Nash equilibrium. Two simulation examples are given to show the effectiveness of the developed optimal control scheme.