High-fidelity implementation of web-based intelligent tutoring system improves fourth and fifth graders content area reading comprehension

•High fidelity implementation is important for web-based intelligent tut. systems.•Effect size .36 on a standardized test of reading comprehension in 4th grade.•Effect size .74–.83 on main idea writing (summary).•Text structure based strategy instruction yields impressive reading outcomes.•Teacher's role in implementation is valuable and cannot be discounted.

Technologies and their effectiveness are impacted by how well they are implemented. A large scale randomized controlled trial was conducted to study the efficacy of a web-based intelligent tutoring system to deliver the structure strategy to improve content area reading comprehension. We present our theory of change focusing on the theoretical framework: structure strategy, delivery approach of web-based intelligent tutoring systems, and contextual conditions for successful adoption of the tool with fidelity. Results from the optimal implementation schools show statistically significantly better performance by ITSS classrooms compared to their control counterparts with moderate to large effect sizes. Conditions for implementing technology-based interventions with fidelity in schools are discussed.