Validation of an ESL writing test in a Malaysian secondary school context

The present study was conducted with a twofold purpose. First, I aim to apply the socio-cognitive framework by Shaw and Weir (2007) in order to validate a summative writing test used in a Malaysian ESL secondary school context. Secondly, by applying the framework I also aim to illustrate practical ways in which teachers can gather validity evidence where this in turn would help them design and evaluate their tests in light of their teaching context and the purpose of assessment. In addition, teachers may be able to reflect on learners’ progress and areas where learners need to improve by looking at the interplay of tasks and learner's response. Twenty exam scripts written by 16-year old ESL learners were rated based on a marking scheme to identify scoring validity. Finally, I will conclude that the validity of score interpretations has been established to a certain degree and the framework is practical for the purposes of the study.

► The study shows that the socio-cognitive framework by Shaw and Weir (2007) is applicable to internal, school-based assessment of L2 writing.
► Validity of interpretation of scores could be achieved when test developers and test users are aware of the purpose the test is intended to serve.
► Designing test tasks related to context and cognitive validity play a crucial role in learners’ responses to the tasks.
► Scoring validity can be achieved with marking schemes, training and standardization.