Review of reliability and treatment integrity trends in autism-focused research

This review summarizes interobserver reliability and treatment integrity trends across five journals in autism-focused single-case research: Focus on Autism and Other Developmental Disabilities, Journal of Applied Behavior Analysis, Journal of Autism and Developmental Disorders, Research in Autism Spectrum Disorder and Education and Training in Autism and Developmental Disabilities. All articles published in the selected journals for the years 1992, 2002, and 2012 were screened based on predetermined inclusion criteria. A total of 119 articles met inclusion criteria and were evaluated in terms of whether they collected reliability data on dependent and independent variables, whether they collected treatment integrity data, as well as the degree to which the data met accepted quality criteria. Trends indicate that the collection and quality of the interobserver agreement data collection for dependent variables is improving. However, the percentage of studies meeting the minimum requirement for collection of interobserver agreement in each phase of the study remains low. Trends for the collection of the treatment integrity data and the quality of the treatment integrity data remains low but is demonstrating an increasing trend. Trends for the interobserver agreement data for the independent variables remain stable and low. Limitations of this review as well as implications for future research are discussed.