Quantifying the development of phraseological competence in L2 English writing: An automated approach

• Proposes an automated technique to assess the phraseological competence of L2 writers.
• Demonstrates that phraseology plays an important role in the development of L2 writing.
• Highlights marked differences between longitudinal and pseudolongitudinal analyses.
• Shows the benefit of phraseological indices that combine lexis and grammar.

Based on the large body of research that shows phraseology to be pervasive in language, this study aims to assess the role played by phraseological competence in the development of L2 writing proficiency and text quality assessment. We propose to use CollGram, a technique that assigns to each pair of contiguous words (bigrams) in a learner text two association scores (mutual information and t-score) computed on the basis of a large reference corpus, the Corpus of Contemporary American English. Applied to the Michigan State University Corpus of second language writing, CollGram shows a longitudinal decrease in the use of collocations made up of high-frequency words that are less typical of native writers. It also shows that the mean MI scores of the bigrams used by L2 writers are positively correlated with the quality of the essays, while there is a negative correlation between the quality of the texts and the proportion of bigrams that were absent in the reference corpus, most of which were shown to be erroneous. The conclusion discusses the marked differences in the effects revealed by the longitudinal and pseudolongitudinal analyses, the limitations of the study, and some potential implications for the teaching and assessment of second language writing.