Finite-horizon optimal control of unknown nonlinear time-delay systems

In this paper, we propose a neural-network (NN)-based online off-policy algorithm to optimize a class of nonlinear continuous-time time-delay systems during finite time horizon. The online off-policy algorithm is used to learn the two-stage solution to the time-varying Hamilton-Jacobi-Bellman (HJB) equation without requiring the knowledge of the time-delay system dynamics. The algorithm is implemented by using an actor-critic NN structure with time-varying activation functions. The weights of the two NNs are tuned simultaneously in real-time by considering both the residual error and the terminal error. Two simulation examples demonstrate the applicability of the proposed algorithm.