Moving beyond the study of gender differences: An analysis of measurement invariance and differential item functioning of an ICT literacy scale

Crafting a validity argument is crucial for the development of any assessment of ICT literacy. In about the context of studying gender differences in ICT literacy, it has therefore become essential to ensure that gender differences are not due to the existence of measurement bias, which might indicate that an assessment instrument used to measure ICT literacy operates differently for girls and boys. Hence, researchers need to gather evidence on the validity of such gender comparisons. The present study follows this line of research by investigating the overall measurement invariance at the construct level and the differential functioning of items across gender of an ICT literacy test. Based on the data obtained from a random sample of 919 Norwegian lower secondary school students (468 girls), multi-group confirmatory factor analysis showed that the test was invariant to a sufficient degree, and girls outperformed boys in the overall test score (β = 0.35, p < 0.001). Yet, differential item functioning existed for selected items. These results highlight the importance of testing for measurement invariance and differential item functioning that goes beyond the mere description of gender differences. Moreover, attention is brought back to the validity of ICT literacy assessments, and ways to improve these assessments are discussed.