Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction

•AWE corrective feedback, together with some instructional methods, may encourage students to engage in revision and writing practice.•AWE corrective feedback may be helpful for ESL writers to improve linguistic accuracy, as indicated by the instructors’ and the students’ perspectives and changes of accuracy observed from student writing.•The instructors’ use of an AWE system varies according to their writing pedagogy and practices of responding to students’ essays, and this may affect students’ use of the system and their evaluation of it.

The development of language processing technologies and statistical methods has enabled modern automated writing evaluation (AWE) systems to provide feedback on language and content in addition to an automated score. However, concerns have been raised with regard to the instructional and assessment value of AWE in writing classrooms. The findings from a few classroom-based studies concerning the impact of AWE on writing instruction and performance are largely inconclusive. Meanwhile, since research provides favorable evidence for the reliability of AWE corrective feedback, and that writing accuracy is both an important and frustrating issue, it is worthwhile to examine more specifically the impact of AWE corrective feedback on writing accuracy. Therefore, the study used mixed-methods to investigate how Criterion® affected writing instruction and performance. Results suggested that Criterion® has led to increased revisions, and that the corrective feedback from Criterion® helped improve accuracy from a rough to a final draft. The potential benefits were also confirmed by the instructors’ interviews. The students’ perspectives were mixed, but the extent to which the views vary may depend on the students’ language proficiency level and their instructors’ use and perspectives of AWE.