Program records as a source for program implementation assessment and youth outcomes predictors during residential care

•Treatment fidelity metrics may be generated using point cards.•Positive youth–staff interactions predict decreased youth externalizing problems.•Positive youth–staff interactions predict decreased non-compliance and school problems.•Intervention generated products may be cost effect and aide in treatment quality.

This study used point card information from a residential program to generate treatment fidelity metrics and determine if the metrics predicted youth outcomes after six months in care. Youth outcomes included staff (n = 52) and youth (n = 143) ratings, youth conduct records kept by the residential program's teaching-family homes and school records. Treatment fidelity metrics included the program components: (a) percentage of positive interactions, (b) number of privileges earned, and (c) a skills taught to interactions ratio. The percentage of positive interactions averaged 90% per youth; 76% of the point cards indicated that privileges were earned; and a variety of life skills were typically taught to the youth (skills ratio = .61). The data from the treatment fidelity metrics supported that the program was implemented consistent with program expectations. The range of implementation quality for each measured component was then used to predict youth outcomes. Increased percent of positive interactions predicted significantly decreased externalizing behaviors as reported by staff (β = − 0.31, p < .001) and youth (β = − 0.30, p < .001), and significantly fewer incidents of non-compliance (Exp(b) = 0.93, p < .001) and school problems (Exp(b) = 0.91, p < .001) as indicated on the program records. The skills ratio indicated similar trends across outcomes, although non-significant at the p < .01 level. Permanent products may be helpful to develop program treatment fidelity metrics, which may be useful for monitoring implementation and may be associated with improved youth outcomes.