Cross-system data linkage for treatment outcome evaluation: Lessons learned from the California Treatment Outcome Project

Using administrative data to evaluate health care outcomes has become increasingly common, but the reliability and validity of outcome measures based on cross-system data linkage have been little scrutinized. Applying a deterministic data matching methodology, we linked 6545 Californians admitted to 43 substance abuse treatment programs between 2000 and 2001 to administrative data acquired from three state agency databases. We compared self-reported treatment outcome measures to equivalent measures derived from official records on motor vehicle driving incidents, criminal history, and mental health services utilization. Administrative data provided analogous results in some measures (e.g., percent of people using mental health services, percent ever arrested) and more accurate results in others (e.g., frequency of service utilization, and frequency of arrests). Similar to findings based on the interview data, the administrative data also revealed improvements in several domains 1-year post-treatment compared to 1-year pre-treatment. Experiences with data linkage procedures and strategies for enhancing record linkage accuracy are discussed. Findings illustrate the value of using administrative records for substance abuse treatment outcome evaluation, while highlighting areas for improvement for future cross-system data linkage efforts.