The Writing Pal Intelligent Tutoring System: Usability Testing and Development

• We describe the development and multiple-method usability testing of an intelligent tutoring system for writing – the Writing Pal.
• Writing Pal offers strategy instruction, educational games, and automated feedback.
• A feasibility study examined the system in multiple high school English classrooms.
• Students perceived Writing Pal as beneficial, informative, and engaging.
• Student and teacher recommendations informed significant system redesign.

The Writing Pal (W-Pal) is an intelligent tutoring system (ITS) designed to improve students’ writing proficiency via a unique combination of explicit strategy instruction, game-based practice, essay writing practice, and automated formative feedback. To develop and refine the many features of the W-Pal tutoring system, we have employed a multiple-method usability testing approach, which capitalizes on the complementary strengths and weakness of methods such as focus groups, component studies, internal testing, and in vivo testing. These diverse methods allow researchers to benefit from focused student input, instructor input, and iterative development, while also gathering data in ecologically-valid settings. In this paper, we describe some of the testing and development of aspects of W-Pal, consider the challenges of building such a system, and provide a particular emphasis on a feasibility study that integrated W-Pal into high school English classrooms during a school year. The results of this study showed that students perceived the system as informative, valuable, and enjoyable, and results also highlighted specific ways that these aspects of the system could be further enhanced. Based upon these findings, a significantly updated version of W-Pal has been developed. The current W-Pal system is described along with considerations for future research and how the system may be used to supplement writing instruction.