Detecting postoperative hemorrhage or hematoma from administrative data: The performance of the AHRQ Patient Safety Indicator

BackgroundPatient Safety Indicator (PSI) 9, “postoperative hemorrhage or hematoma” (PHH), of the US Agency for Healthcare Research and Quality has been considered for public quality of care reporting. We sought to evaluate its performance in detecting true complications.MethodsWe conducted a retrospective, cross-sectional study of hospitalizations that met PSI 9 eligibility criteria. We sampled records flagged positive and negative by PSI 9 from a diverse set of 31 hospitals between February 2006, and June 2009. Trained abstractors reviewed medical records using standard instruments. We determined the sensitivity, specificity, and positive (PPV) and negative (NPV) predictive values of the indicator.ResultsOf 181 analyzable records flagged by PSI 9, 168 (93%; weighted PPV, 95% [95% confidence interval (CI), 90–98%]) involved an accurately coded event, but only 126 (70%; weighted PPV, 78% [95% CI, 58–90%]) represented true PHH. Thirty-two false positives involved only intraoperative hemorrhage. Among true positives, hypotension occurred in 28% and death attributed to the PHH in 4%. Thirty-two of 281 records flagged negative by PSI 9 (but enriched with questionably negative records) represented true PHH. The indicator's sensitivity was 42% (95% CI, 23–64%), specificity 99.9% (95% CI, 99.8–100%), and NPV 99.7% (95% CI, 99.0–99.9%). Modifying the indicator to include additional procedure codes improved both sensitivity (85% [95% CI, 67–94%]) and PPV (76% [95% CI, 60–88%]).ConclusionPSI 9 holds promise in detecting serious, possibly preventable complications. The indicator might be improved by specification of the 998.11 hemorrhage code to exclude purely intraoperative events and addition of procedure codes to the indicator's numerator criteria.