کد مقاله کد نشریه سال انتشار مقاله انگلیسی نسخه تمام متن
412673 679676 2013 10 صفحه PDF دانلود رایگان
عنوان انگلیسی مقاله ISI
Transfer learning with Partially Constrained Models: Application to reinforcement learning of linked multicomponent robot system control
موضوعات مرتبط
مهندسی و علوم پایه مهندسی کامپیوتر هوش مصنوعی
پیش نمایش صفحه اول مقاله
Transfer learning with Partially Constrained Models: Application to reinforcement learning of linked multicomponent robot system control
چکیده انگلیسی

Transfer learning is a hierarchical approach to reinforcement learning of complex tasks modeled as Markov Decision Processes. The learning results on the source task are used as the starting point for the learning on the target task. In this paper we deal with a hierarchy of constrained systems, where the source task is an under-constrained system, hence called the Partially Constrained Model (PCM). Constraints in the framework of reinforcement learning are dealt with by state-action veto policies. We propose a theoretical background for the hierarchy of training refinements, showing that the effective action repertoires learnt on the PCM are maximal, and that the PCM-optimal policy gives maximal state value functions. We apply the approach to learn the control of Linked Multicomponent Robotic Systems using Reinforcement Learning. The paradigmatic example is the transportation of a hose. The system has strong physical constraints and a large state space. Learning experiments in the target task are realized over an accurate but computationally expensive simulation of the hose dynamics. The PCM is obtained simplifying the hose model. Learning results of the PCM Transfer Learning show an spectacular improvement over conventional Q-learning on the target task.


► Transfer learning based on Partially Constrained Models for speeding up reinforcement learning in highly constrained systems.
► Proof of optimality of the transfer learning: effective action repertoires and optimal policies.
► Application to a hose manipulation problem.
► Experimental results on an accurate simulation model show great improvement of learning speed over conventional Q-learning.

ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: Robotics and Autonomous Systems - Volume 61, Issue 7, July 2013, Pages 694–703
نویسندگان
, , ,