A two-layer networked learning control system using actor–critic neural network

Aiming at the fact that the controlled objects are becoming more and more complex, while the existing control strategies cannot fulfill the needs for higher quality control performance due to the limitations of the widely used single-layer networked control system architecture, a two-layer networked learning control system architecture is proposed. Under this architecture, independent local controllers are interconnected to form the lower layer, while a learning agent which communicates with the independent local controllers in the lower layer forms the upper layer. To implement such a system, a discard-packet strategy is firstly developed to deal with network-induced delay, data packet out-of-order and data packet loss. The cubic spline interpolator is then employed to compensate the lost data. Finally, the output of the learning agent using actor–critic neural network is used to dynamically tune the control signal of local controller. Control simulations of different ways for a nonlinear heating, ventilation and air-conditioning (HVAC) system are compared. Simulation results show that this new architecture can effectively improve the control performance.