The (in) comparability of ICT knowledge and skill self-assessments among upper secondary school students: The use of the anchoring vignette method

• We compare the ICT skill self-assessments of ICT and non-ICT students.
• Self-reports do not correspond to the actual level of students' ICT skills.
• We identify the heterogeneity in students' response style.
• The anchoring vignette method adjusts the self-reports for the heterogeneity.
• Adjusted self-reports correspond to the actual level of students' ICT skills.

Self-reporting of ICT knowledge and skills is commonly used in questionnaire surveys among students and in job search situations to provide information about actual ICT knowledge and skills. The main advantage of this approach is its low cost. The responses can, however, reflect not only the actual level of knowledge and skills but also the self-assessment style. Two students with the same actual knowledge and skills level may give different self-assessments since one might be optimistic and overestimate his/her skills and the other could be pessimistic and underestimate his/her skills. The anchoring vignette method helps to adjust self-reports for the differences in scale usage. We compare the ICT knowledge and skills of two distinct groups of Czech upper secondary school students, ICT students (N = 228) and non-ICT (business and pedagogy) students (N = 147), based on their self-reported general ICT knowledge and skills, before and after the anchoring vignette adjustment for the different usage of scale. We show that the anchoring vignette method helps to distinguish between the two groups of students' differences in scale usage and that adjusted self-reports correspond to the actual level of students' ICT knowledge and skills (in contrary to the unadjusted self-reports). The anchoring vignette method is a very promising tool for increasing the validity of surveys that use students' ICT knowledge and skill self-reports.