Optimal tracking control for completely unknown nonlinear discrete-time Markov jump systems using data-based reinforcement learning method

In this paper, we develop a novel optimal tracking control scheme for a class of nonlinear discrete-time Markov jump systems (MJSs) by utilizing a data-based reinforcement learning method. It is not practical to obtain accurate system models of the real-world MJSs due to the existence of abrupt variations in their system structures. Consequently, most traditional model-based methods for MJSs are invalid for the practical engineering applications. In order to overcome the difficulties without any identification scheme which would cause estimation errors, a model-free adaptive dynamic programming (ADP) algorithm will be designed by using system data rather than accurate system functions. Firstly, we combine the tracking error dynamics and reference system dynamics to form an augmented system. Then, based on the augmented system, a new performance index function with discount factor is formulated for the optimal tracking control problem via Markov chain and weighted sum technique. Neural networks are employed to implement the on-line ADP learning algorithm. Finally, a simulation example is given to demonstrate the effectiveness of our proposed approach.