Evidence on the effects of task interactivity and grade level on thinking skills involved in complex problem solving

•We developed a reliable measure of complex problem solving in science.•A three-dimensional model substantially represented the data.•Task interactivity significantly affected students’ problem-solving performance.•Students of grade 10 outperformed eighth graders in analytical tasks.•Fluid intelligence and problem solving were significantly correlated.

The present study aims to analyze the structure of problem solving abilities which can be regarded as higher-order thinking skills in the domain of science. We investigate the effects of task interactivity and grade level on problem solving and check whether a method factor of task interactivity improved our proposed model. 805 high-school students of grades 8 and 10 worked on a computer-based assessment and completed tests on related constructs such as intelligence, domain knowledge, and strategy knowledge in science. Using confirmatory factor analysis, we established a measurement model with three correlated traits, one method factor of interactivity, and grade level as a predictor. Our results suggest that: (1) scientific problem solving can be regarded as a multidimensional construct, (2) task interactivity is a substantial factor in determining students’ problem-solving success, and (3) there is a development within the analytical component of problem solving across grades. We conclude that psychological theories of problem solving and interactivity can be transferred to complex problem-solving situations in the domain of science.