Why fifth- and seventh-graders submit off-task responses to a web-based reading comprehension tutor rather than expected learning responses

•5th & 7th Graders who submitted off-task responses in the system displayed lower scores on learning/mastery goals.•Highest submitters of off-task responses in the on-line reading instruction had low motivation to read.•Students with the most off-task responses scored over 2 SD below average on on-line practice tasks.•Motivational scales prior to instruction could flag students at risk for disengagement with on-line learning.•On-line performance measures could route students to lessons for prevention of off-task behaviors.

Research shows the students improve their reading comprehension with Intelligent Tutoring of the Structure Strategy (ITSS). One problem for ITSS is that some students are producing responses in the on-line instruction that are unrelated to learning and practicing the reading strategy. These types of disengaged responses can be referred to as system active off-task responses (“off-task”). In this study we characterize who produces off-task responses and why. Classification and Regression Trees (C&RT) and logistic regression analyses were used to answer the why question. Variables predicted to relate to gaming included reading strategy and skill variables, motivation, attitude, self-efficacy, and goal orientation variables, demographic variables, and type of computer feedback (simple versus elaborated). C&RT analysis could explain 66% of the variance in off-task responses. Students without off-task responses were higher in motivation to read and worked in ITSS to produce good main ideas. Students with higher off-task responses had low scores on work mastery goals. The highest producers of off-task responses in Grades 5 and 7 (averaging 24 off-task responses over 7 lessons) had low motivation to read and scored over 2 SD below average on recall tasks in ITSS. The logistic regression could explain 42% of the variance in off-task responses. Use of motivational scales prior to starting instruction as well as on-line performance measures could be used to flag students for early intervention to prevent system active off-task responses and increase on-line learning. The C&RT approach may be particularly helpful to designers in making software more appropriate for different types of students.