The effects of computer-generated feedback on the quality of writing

• Most of the research focuses on the effects of AWE feedback on written production.
• Modest evidence was found that AWE feedback has a positive effect on the quality of the AWE texts.
• As yet, there is little evidence that effects of AWE feedback are associated with more general improvements in writing.

This study provides a critical review of research into the effects of computer-generated feedback, known as automated writing evaluation (AWE), on the quality of students’ writing. An initial research survey revealed that only a relatively small number of studies have been carried out and that most of these studies have examined the effects of AWE feedback on measures of written production such as scores and error frequencies. The critical review of the findings for written production measures suggested that there is modest evidence that AWE feedback has a positive effect on the quality of the texts that students produce using AWE, and that as yet there is little evidence that the effects of AWE transfer to more general improvements in writing proficiency. Paucity of research, the mixed nature of research findings, heterogeneity of participants, contexts and designs, and methodological issues in some of the existing research were identified as factors that limit our ability to draw firm conclusions concerning the effectiveness of AWE feedback. The study provides recommendations for further AWE research, and in particular calls for more research that places emphasis on how AWE can be integrated effectively in the classroom to support writing instruction.