Evidence on the effects of task interactivity and grade level on thinking skills involved in complex problem solving

• We developed a reliable measure of complex problem solving in science.
• A three-dimensional model substantially represented the data.
• Task interactivity significantly affected students’ problem-solving performance.
• Students of grade 10 outperformed eighth graders in analytical tasks.
• Fluid intelligence and problem solving were significantly correlated.

The present study aims to analyze the structure of problem solving abilities which can be regarded as higher-order thinking skills in the domain of science. We investigate the effects of task interactivity and grade level on problem solving and check whether a method factor of task interactivity improved our proposed model. 805 high-school students of grades 8 and 10 worked on a computer-based assessment and completed tests on related constructs such as intelligence, domain knowledge, and strategy knowledge in science. Using confirmatory factor analysis, we established a measurement model with three correlated traits, one method factor of interactivity, and grade level as a predictor. Our results suggest that: (1) scientific problem solving can be regarded as a multidimensional construct, (2) task interactivity is a substantial factor in determining students’ problem-solving success, and (3) there is a development within the analytical component of problem solving across grades. We conclude that psychological theories of problem solving and interactivity can be transferred to complex problem-solving situations in the domain of science.