Development of an online test of problem solving ability that minimises the extraneous differential effects of language background and subsequent validation through a large-scale DIF analysis

• We present the development of a test of problem solving ability designed to minimise effects of language background.
• The test was validated using differential item functioning analysis.
• This has important implications for Australia's linguistically diverse classrooms.

This paper presents a test of problem solving ability that is designed to minimise the differential effects of language background. Large-scale data are presented in order to determine the validity of this test through differential item functioning (DIF) analysis. The analysis was conducted on test data from 3527 students from 61 government schools across Victoria, Australia in 2010. The students have diverse linguistic backgrounds and a wide range of English language proficiency.Using both 2PL IRT and logistic regression approaches it is shown that most items are not functioning differentially with respect to language background, providing evidence that the design of the instrument is validated by the data. The implications for future system-wide implementation of large scale interventions for Australia's linguistically diverse classrooms are outlined and discussed.