Development and validation of an automated delirium risk assessment system (Auto-DelRAS) implemented in the electronic health record system

BackgroundA key component of the delirium management is prevention and early detection.ObjectiveTo develop an automated delirium risk assessment system (Auto-DelRAS) that automatically alerts health care providers of an intensive care unit (ICU) patient's delirium risk based only on data collected in an electronic health record (EHR) system, and to evaluate the clinical validity of this system.DesignCohort and system development designs were used.SettingMedical and surgical ICUs in two university hospitals in Seoul, Korea.ParticipantsA total of 3284 patients for the development of Auto-DelRAS, 325 for external validation, 694 for validation after clinical applications.MethodsThe 4211 data items were extracted from the EHR system and delirium was measured using CAM-ICU (Confusion Assessment Method for Intensive Care Unit). The potential predictors were selected and a logistic regression model was established to create a delirium risk scoring algorithm to construct the Auto-DelRAS. The Auto-DelRAS was evaluated at three months and one year after its application to clinical practice to establish the predictive validity of the system.ResultsEleven predictors were finally included in the logistic regression model. The results of the Auto-DelRAS risk assessment were shown as high/moderate/low risk on a Kardex screen. The predictive validity, analyzed after the clinical application of Auto-DelRAS after one year, showed a sensitivity of 0.88, specificity of 0.72, positive predictive value of 0.53, negative predictive value of 0.94, and a Youden index of 0.59.ConclusionsA relatively high level of predictive validity was maintained with the Auto-DelRAS system, even one year after it was applied to clinical practice.