Local Models for data-driven learning of control policies for complex systems

An approach based on local learning, relying on Nadaraya–Watson models (NWMs), is introduced for the problem of deriving an automatic controller able to exploit data collected during the operation of some complex plant or system by a reference teacher (e.g., a human operator). Such learning approach is particularly useful when the system is too complex to be modeled accurately and/or the task cannot be easily formalized by a cost function, a situation which rules out classic approaches based, e.g., on dynamic programming. Here it is proved that local models are a suitable solution for a real-time employment, since they allow to incorporate new information directly and efficiently without the need of offline training, and new data immediately reflect in improvement of performance. To this purpose, convergence analysis of the method is provided, also considering the case where the reference controller introduces random variations in the training data. Finally, a simulation test, concerning the control of a mechanical system, is provided to showcase the use of local models in an applicative scenario.

► We propose local models to learn the behavior of a reference controller.
► The method relies on data collected during successful operations of the system.
► The proposed models exploit new data in a straightforward way.
► Local models of the Nadaraya–Watson kind allow convergence to the reference policies.
► A simulation example shows the effectiveness of the proposed method.