Putting testing researchers to the test: An exploratory study on the TOEFL iBT

Despite being one of the most widely used proficiency measures of English for Academic Purposes, the newly-designed Internet-based Test of English as a Foreign Language (TOEFL iBT) remains externally under researched compared to previous TOEFL versions. Specifically, research is needed on factors that effect student performance within this new testing context. The purpose of this research was to identify and raise potential issues associated with the TOEFL iBT as explicitly linked to construct-dependent and construct-irrelevant variance factors. Through a key informant method, four language testing researchers sat for the TOEFL iBT and reported their experiences through two externally mediated focus groups. The testing researchers' experiences were analyzed using a standard thematic analysis then deductively grouped based on their form of measurement variance. Results provide valuable considerations for measuring English for academic purposes and serve to identify specific, practical issues related to the language testing conditions, question design, and the testing protocol of the TOEFL iBT.