Using agency data for evidence-based programming: A university–agency collaboration

Agencies providing residential treatment are encouraged, or even mandated, to collect outcomes data and to implement evidence based practices, yet little guidance has been provided on how to do so using agency administrative data that are collected on an ongoing basis. We examined data on Child and Adolescent Functional Assessment Scale (CAFAS) scores for 1608 admissions to a residential treatment center from 2002 through 2008. CAFAS scores were measured every 90 days, providing multiple CAFAS scores for each individual. Results demonstrated that on average residents improved in functioning over time. Sensitive to the evolving needs of residents who had been entering the program, the treatment center made significant program changes in 2006 to attempt to better serve residents through a broad array of specialized programming. Compared to the overall results, the analysis suggested that residents who entered the program since October 2006 appeared to have made larger improvements in their CAFAS scores. Results were derived by employing multilevel models appropriate for estimating growth trajectories with repeated measures data. Conversations with agency staff suggested that using administrative data, and advanced statistical models, were extremely helpful for organizational decision making and evidence-based programming.

► We model how to use administrative data to inform organizational decision making.
► Growth trajectories were derived by employing multilevel longitudinal models.
► Repeated measures CAFAS data was found to be robust for analysis.
► Differential effects as a result of evidence-based programming were demonstrated.