Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator for Postoperative Respiratory Failure (PSI 11) does not identify accurately patients who received unsafe care

BackgroundThe Agency for Healthcare Research and Quality Patient Safety Indicator 11 is used to identify postoperative respiratory failure events and detect areas for quality improvement. This study examines the accuracy of Patient Safety Indicator 11 in identifying clinically valid patient safety events.MethodsAll cases flagged for Patient Safety Indicator 11 from July 2013 to July 2015 by Agency for Healthcare Research and Quality QI Version 4.5 including International Classification of Diseases–9 codes were evaluated. Code-confirmed cases underwent independent review by 2 physicians. Inpatient electronic medical records were used to identify clinical factors for postoperative respiratory failure in each case to determine if postoperative respiratory failure was a result of unsafe care. The clinical true-positive rate and positive predictive value were calculated.ResultsA total of 166 postoperative respiratory failure cases were reviewed; 51 were recoded and reversed due to coding or documentation errors; 115 cases met the Agency for Healthcare Research and Quality definition of postoperative respiratory failure. A total of 71 (61.7%) of the 115 cases were false positives and did not reflect unsafe care, while 44 cases were true positives with a positive predictive value of 38.3%. χ2 analysis did not reveal an association between demographics, clinical characteristics, or operative procedure with true-positive cases.ConclusionAdministrative coding data for Agency for Healthcare Research and Quality Patient Safety Indicator 11 do not identify accurately patients who received unsafe care when taking into account unpreventable clinical factors causing postoperative respiratory failure. The use of Agency for Healthcare Research and Quality Patient Safety Indicator 11 as a hospital performance measure should be reconsidered until inclusion and exclusion criteria are revised.