An iterative ϵϵ-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state

In this paper, a finite horizon iterative adaptive dynamic programming (ADP) algorithm is proposed to solve the optimal control problem for a class of discrete-time nonlinear systems with unfixed initial state. A new ϵϵ-optimal control algorithm based on the iterative ADP approach is proposed that makes the performance index function iteratively converge to the greatest lower bound of all performance indices within an error ϵϵ in finite time. The convergence analysis of the proposed ADP algorithm in terms of performance index function and control policy is conducted. The optimal number of control steps can also be obtained by the proposed ϵϵ-optimal control algorithm for the unfixed initial state. Neural networks are used to approximate the performance index function, and compute the optimal control policy, respectively, for facilitating the implementation of the ϵϵ-optimal control algorithm. Finally, a simulation example is given to show the effectiveness of the proposed method.