H∞ control with constrained input for completely unknown nonlinear systems using data-driven reinforcement learning method

This paper investigates the H∞ control problem for nonlinear systems with completely unknown dynamics and constrained control input by utilizing a novel data-driven reinforcement learning method. It is known that nonlinear H∞ control problem relies on the solution of Hamilton-Jacobi-Isaacs (HJI) equation, which is essentially a nonlinear partial differential equation and generally impossible to be solved analytically. In order to overcome this difficulty, firstly, we propose a model-based simultaneous policy update algorithm to learn the solution of HJI equation iteratively and provide its convergence proof. Then, based on this model-based method, we develop a data-driven model-free algorithm, which only requires the real system sampling data generated by arbitrary different control inputs and external disturbances instead of accurate system models, and prove that these two algorithms are equivalent. To implement this model-free algorithm, three neural networks (NNs) are employed to approximate the iterative performance index function, control policy and disturbance policy, respectively, and the least-square approach is used to minimize the NN approximation residual errors. Finally, the proposed scheme is tested on the rotational/translational actuator nonlinear system.