Bringing reading-to-write and writing-only assessment tasks together: A generalizability analysis

Integrated tasks are currently employed in a number of L2 exams since they are perceived as an addition to the writing-only task type. Given this trend, the current study investigates composite score generalizability of both reading-to-write and writing-only tasks. For this purpose, a multivariate generalizability analysis is used to investigate the effect of both the task and rater facets on composite score reliability. In addition, a comparison of two different rating schemes is presented—two different groups of raters scoring each task type versus the same raters scoring both task types. The researcher collected data from 115 examinees who completed two writing-only and two reading-to-write tasks. mGENOVA (Brennan, R.L. (1999). Manual for mJENOVA (version 2.0). Iowa City, IA: CASMA) a program mainly designed for multivariate generalizability was used to analyze the study data. The results showed that a composite score of both task types is as reliable as scores obtained from either writing-only or reading-to-write tasks. The results also demonstrated that having different raters score each task type would produce reliable scores similar to those obtained from having the same raters score both task types. Implications and limitations of the study are presented.