Direct measures of digital information processing and communication skills in primary education: Using item response theory for the development and validation of an ICT competence scale

In the past decade, several studies have measured ICT competences from the perspective of ICT self-efficacy. Such indirect measurements tend to have validity problems, as they depend on the pupils' ability to judge their own ICT competences. This study outlines the development of a performance-based digital test and the validation of a direct measure of ICT competence through the use of item response theory (IRT). More specifically, the test and the developed measure focus on primary-school pupils' proficiency in digital information processing and communication. 56 Items were administered to 560 pupils at the end of their primary-school education (age between 10.79 and 13.85 years old). The items were controlled for dimensionality, model-data fit, local item dependence and monotonicity. The final measure contains 27 items that refer to retrieving and processing digital information, and communication with a computer. The results indicate that the instrument is particularly reliable for low and median ability levels. Further refinement and possible future use of the instrument is discussed.