An investigation into current measures of linguistic accuracy in second language writing research

• Accuracy measures in current L2 writing research are classified and discussed.
• A variety of accuracy measures were used on the data set.
• Reliability and validity of the measures are discussed.
• Measures focusing solely on accuracy did not show improvement over time.

This study begins by describing measures of linguistic accuracy in second language writing research. We first report on measures that have been used in the last ten years including holistic measures, error-free units, number of errors, number of specific error types, and measures that take error severity into account. We discuss differences in the measures used now vs. those reported on in Polio (1997) and conclude that interrater reliability and detailed coding guidelines are still underreported, making replication of studies very difficult. We then apply ten of these measures to the MSU data set to determine which are the most reliable and which best show change over time. The measures comprise holistic and error-free units as well as counts of specific error types. With the exception of counts of certain error types, we were able to obtain over .80 reliability on the measures though some of the measures were much easier to use than others. In addition, we attempted to determine whether or not these measures were measuring the same or different constructs by looking at correlations and change over time. Among the correlations, some were expected and some not. One conclusion was that weighted error-free units did not seem to differ from standard error-free units. Only the holistic measures and the number of preposition errors showed any change over time.