Online optimal control of unknown discrete-time nonlinear systems by using time-based adaptive dynamic programming

In this paper, an online optimal control scheme for a class of unknown discrete-time (DT) nonlinear systems is developed. The proposed algorithm using current and recorded data to obtain the optimal controller without the knowledge of system dynamics. In order to carry out the algorithm, a neural network (NN) is constructed to identify the unknown system. Then, based on the estimated system model, a novel time-based ADP algorithm without using system dynamics is implemented on an actor-critic structure. Two NNs are used in the structure to generate the optimal cost and the optimal control policy, and both of them are updated once at the sampling instant and thus the algorithm can be regarded as time-based. The persistence of excitation condition, which is generally required in adaptive control, is ensured by a new criterion while using current and recorded data in the update of the critic neural network. Lyapunov techniques are used to show that system states, cost function and control signals are all uniformly ultimately bounded (UUB) with small bounded errors while explicitly considering the approximation errors caused by the three NNs. Finally, simulation results are provided to verify the effectiveness of the proposed approach.