Exploring individual and item factors that affect assessment validity for diverse learners: Results from a large-scale cognitive lab

A cognitive lab technique (n = 156) was used to investigate interactions between individual factors and item factors presumed to affect assessment validity for diverse students, including English language learners. Findings support the concept of access — an interaction between specific construct-irrelevant item features and individual characteristics that either permits or inhibits student response to the targeted measurement content of an assessment item. Access issues of 3rd and 5th grade students were explored at three stages during problem solving on mathematics items: 1) apprehension of task demands; 2) formulation of a solution; and 3) articulation of a solution. Adequacy of access at these levels appears to affect student performance. In particular, where students were able to experience increased apprehension in the first stage through the provision of item variations consistent with their individual access needs, they were more likely to formulate correct solution strategies. The implications of these findings are discussed and suggestions for future research are offered.