Closed-loop control of anesthesia and mean arterial pressure using reinforcement learning

•Closed-loop anesthesia controller using reinforcement learning (RL) algorithm.•Q-learning algorithm for the control of multiple parameters in dynamical systems.•The proposed method is tested on 30 randomized simulated patients.•Simulations demonstrate comparable performance with the recent clinical trials conducted.

General anesthesia is required for some patients in the intensive care units (ICUs) with acute respiratory distress syndrome. Critically ill patients who are assisted by mechanical ventilators require moderate sedation for several days to ensure cooperative and safe treatment in the ICU, reduce anxiety and delirium, facilitate sleep, and increase patient tolerance to endotracheal tube insertion. However, most anesthetics affect cardiac and respiratory functions. Hence, it is important to monitor and control the infusion of anesthetics to meet sedation requirements while keeping patient vital parameters within safe limits. The critical task of anesthesia administration also necessitates that drug dosing be optimal, patient specific, and robust. In this paper, the concept of reinforcement learning (RL) is used to develop a closed-loop anesthesia controller using the bispectral index (BIS) as a control variable while concurrently accounting for mean arterial pressure (MAP). In particular, the proposed framework uses these two parameters to control propofol infusion rates to regulate the BIS and MAP within a desired range. Specifically, a weighted combination of the error of the BIS and MAP signals is considered in the proposed RL algorithm. This reduces the computational complexity of the RL algorithm and consequently the controller processing time.