Measuring Students’ Progressions in Scientific Problem Solving: A Psychometric Approach

The ability to solve complex problems is one of the key competencies in science. But until now, little research on students’ progressions in scientific problem solving has been conducted. This study addresses the following research question: Which factors can be distinguished in order to describe the structure of the ability to solve scientific problems and how does this ability develop across grade levels? Within a cross-sectional survey, we used computer-based assessment tools to capture students’ problem-solving abilities in grades 8, 10, and 12 (N=1,487). Based on four key dimensions, a vertical scale has been established by using an IRT modeling approach. This model was tested for measurement invariance as a prerequisite for comparing different grade levels and was, finally, validated by multilevel regression analyses. Our results showed that the ability to solve interactive scientific problems can be described by four cognitive factors. Based on a vertical scale, this structure held across grade levels and revealed significant progressions. Finally, different developmental patterns were found, which were related to reasoning, strategy knowledge, and domain knowledge. We conclude that our model of scientific problem solving can be used to capture students’ interindividual progressions.