کد مقاله کد نشریه سال انتشار مقاله انگلیسی نسخه تمام متن
695465 1460665 2014 10 صفحه PDF دانلود رایگان
عنوان انگلیسی مقاله ISI
Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design
ترجمه فارسی عنوان
تکرار سیاست تقریبی سیاست مبتنی بر داده ها برای کنترل بهینه کنترل نشده غیرخطی
کلمات کلیدی
کنترل بهینه غیر خطی، تقویت یادگیری، خارج از سیاست، تکرار سیاست تقریبی مبتنی بر داده، شبکه عصبی، معادله همیلتون-یعقوبی-بلمن
موضوعات مرتبط
مهندسی و علوم پایه سایر رشته های مهندسی کنترل و سیستم های مهندسی
چکیده انگلیسی

This paper addresses the model-free nonlinear optimal control problem based on data by introducing the reinforcement learning (RL) technique. It is known that the nonlinear optimal control problem relies on the solution of the Hamilton–Jacobi–Bellman (HJB) equation, which is a nonlinear partial differential equation that is generally impossible to be solved analytically. Even worse, most practical systems are too complicated to establish an accurate mathematical model. To overcome these difficulties, we propose a data-based approximate policy iteration (API) method by using real system data rather than a system model. Firstly, a model-free policy iteration algorithm is derived and its convergence is proved. The implementation of the algorithm is based on the actor–critic structure, where actor and critic neural networks (NNs) are employed to approximate the control policy and cost function, respectively. To update the weights of actor and critic NNs, a least-square approach is developed based on the method of weighted residuals. The data-based API is an off-policy RL method, where the “exploration” is improved by arbitrarily sampling data on the state and input domain. Finally, we test the data-based API control design method on a simple nonlinear system, and further apply it to a rotational/translational actuator system. The simulation results demonstrate the effectiveness of the proposed method.

ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: Automatica - Volume 50, Issue 12, December 2014, Pages 3281–3290
نویسندگان
, , , ,