Distributed adaptive dynamic programming for data-driven optimal control

Adaptive dynamic programming (ADP), as an important optimal control technique, can be exploited in the setting of data-driven control based on an approximate regression-based solution of the Hamilton-Jacobi-Bellman (HJB) equations. Distributed optimization algorithms, which are extensively studied in statistics and machine learning, have not yet been applied to the solution of data-driven ADP problems. In this work, we identify the data-driven ADP problem as a consensus optimization problem for nonlinear affine systems, and apply the alternating direction method of multipliers (ADMM) and its accelerated variants for its solution. For the input-constrained optimal control problem, we define a combined optimal primal-dual function to develop a data-based version of the input-constrained HJB equation.