Sometimes less is more: Comparing the validity of complex problem solving measures

The world we live in is becoming increasingly connected and dynamic. With this increase in complexity in people's everyday lives, there is a rising demand for instruments assessing the skills necessary to deal with complex situations. Therefore, both researchers and practitioners have frequently used measures of complex problem solving skills (CPS) since their development in the 1970s. These classical measures of CPS have been questioned because of their insufficient psychometric qualities. In response, newer measures of CPS with a stronger focus on psychometric qualities were developed. However, research comparing the validity of classical measures of CPS to the new measures is limited. The aim of this study is to compare classical measures of CPS to measures following the multiple complex systems approach (MCS), a new approach towards CPS assessment, with regard to their validity. Based on a university student sample (N = 339), we compared the Tailorshop, an established classical measure of CPS to three different MCS tests (MicroDYN, Genetics Lab, MicroFIN). Examining construct validity, MCS tests correlated significantly higher with each other than with the Tailorshop and this difference was not attributable exclusively to a higher share of reasoning within the MCS tests. Further, in predicting school grades, MCS tests were found to have incremental validity over the Tailorshop but not over reasoning and the Tailorshop combined. These findings provide further empirical support for the proposition of MCS tests as valid alternatives to classical measures for the assessment of CPS and add to the literature on the relation between CPS and intelligence.