Informatics and the American College of Surgeons National Surgical Quality Improvement Program: Automated Processes Could Replace Manual Record Review

BackgroundThe American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) provides reliable, risk-adjusted outcomes data using standardized definitions and end points. Collection of the data is time consuming, and the surgical clinical nurse reviewers (SCNRs) can sample only a subset of all surgical cases. We sought to test the feasibility of using an informatics tool to automatically identify postoperative complications stored as free-text documents in our electronic medical record.Study DesignWe used a locally developed electronic medical record search engine (EMERSE) to build sets of terminology that could accurately identify postoperative complications of both myocardial infarction (MI) and pulmonary embolism (PE) as defined by the ACS-NSQIP. All complications had been previously identified by our SCNRs and these were considered the gold standard. We used 5,894 cases from 2001 to 2004 from our institution's ACS-NSQIP dataset for building the terminology and 4,898 cases from 2005 to 2006 for validation. False-positive cases were then further reviewed manually.ResultsWe achieved sensitivities of 100.0% and 92.8% for identifying postoperative myocardial infarction and pulmonary embolism, respectively, with somewhat lower specificities of 93.0% and 95.9%, respectively. These results compared favorably with results from the SCNRs, especially because our manual review uncovered cases previously missed.ConclusionsInformatics has the potential to improve the efficiency and accuracy of chart abstraction by SCNRs for the ACS-NSQIP. Using such tools may eventually allow all cases at an institution to be reviewed rather than a small subset.