Taking a future perspective by learning from the past – A systematic review of assessment instruments that aim to measure primary and secondary school students' ICT literacy

•We review empirical studies that aim to measure students' ICT literacy.•Most tests focus on lower secondary students' ICT literacy.•Some dimensions of ICT literacy are scarcely measured.•A lack of consistent reporting of the reliability and validity indicators exists.•This study provides the state-of-the-art of ICT literacy assessments in education.

This study systematically reviews literature on assessment instruments of primary and secondary school students' ICT literacy. It has three objectives: (1) Describe the development and characteristics of the assessments; (2) Present a synthesis of the facets of ICT literacy measured; and (3) Investigate to what extent information about reliability and validity is provided. A total of 38 tests reported in 66 studies were included. The results indicate that most of the tests target lower secondary students, comprise multiple-choice item designs, and are evaluated by quantitative methodology. The majority of the tests measure facets such as searching, retrieving, and evaluating digital information, and technical skills. In particular, the access to tests measuring digital communication, collaboration, safety, and problem solving is limited. This review demonstrates that an adequate norm for documenting and reporting test quality is lacking. Our findings point to potential future directions in developing and reporting assessments of ICT literacy.