Examining the Specification Validity of the HEDIS Quality Measures for Substance Use Disorders

•Accurate operationalization is a major challenge in developing quality measures.•Specification validity (SV) is whether a quality measure is accurately operationalized.•HEDIS® SUD Initiation and Engagement measures had good SV in SUD programs.•For data from outside SUD programs, SV was somewhat lower and highly variable.

Accurate operationalization is a major challenge in developing quality measures for substance use disorder treatment. Specification validity is a term used to describe whether a quality measure is operationalized such that it captures the intended care processes and patients. This study assessed the specification validity of the 2009 Healthcare Effectiveness Data and Information Set (HEDIS®) substance use disorder initiation and engagement measures by examining whether encounters assumed to include relevant treatment have corroborating evidence in the clinical progress notes. The positive predictive values were excellent (> 90%) for residential and outpatient records selected from addiction treatment programs but more modest for records generated in non-addiction settings, and were highly variable across facilities. Stakeholders using these measures to compare care quality should be mindful of the clinical composition of the data and determine if similar validation work has been conducted on the systems being evaluated.