Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study

•We investigated if linguistic features predict L2 writing proficiency in integrated and independent writing tasks.•We examined differences in the linguistic features between the two tasks that are predictive of writing proficiency.•Linguistic variables explained 53–57% of the variance in writing proficiency.•Syntactic features positively predicted performance on the integrated task.•Cohesion features negatively predicted performance on the independent task.

This study explores whether linguistic features can predict second language writing proficiency in the Test of English as a Foreign Language (TOEFL iBT) integrated and independent writing tasks and, if so, whether there are differences and similarities in the two sets of predictive linguistic features. Linguistic features related to lexical sophistication, syntactic complexity, cohesion, and basic text information were investigated in relation to the writing scores for both integrated and independent samples. The results of this study show that linguistic features can be used to significantly predict essay scores in the integrated as well as the independent writing. When comparing across the two writing tasks, there are both similarities and differences in the two sets of predictive features. For instance, lexical sophistication was found to be a significant predictor for both tasks while features such as verbs in 3rd person singular form and semantic similarity were only significant predictors for the integrated task. These findings demonstrate that evaluation of the two writing tasks rely on similar and distinct features, and are at least partially assessed using different linguistic criteria. Implications of these findings for the assessment of second language (L2) writing are also discussed.