Comparability of essay question variants

Writing task variants can increase test security in high-stakes essay assessments by substantially increasing the pool of available writing stimuli and by making the specific writing task less predictable. A given prompt (parent) may be used as the basis for one or more different variants. Six variant types based on argument essay prompts from a large-scale, high-stakes North American writing assessment and six based on issue prompts from the same test were created and evaluated in the research section of the test administrations in the winter of 2009. Examinees were asked to volunteer to write an essay on one of the new prompt/variant tasks. Essays were obtained from 7573 examinees for argument prompts and 10,827 examinees for issue prompts. Results indicated that all variant types produced reasonably similar means, standard deviations, and rater reliabilities, suggesting that the variant strategy should be useable for operational administrations in high stakes essay assessments. Variant type did not interact with gender, ethnicity, or language (self-report that English or another language is the examinee's “best” language).

► The task of creating essay questions can be eased by developing many different types of questions (variants) that are based on a single prompt. ► Variants can be created that are of approximately equal difficulty, produce similar distributions of essay scores, and can be rated with equal reliability. ► No variant type is differentially difficult for a particular gender, ethnic, or English experience (English first language or other language first) group.