Problematising the equivalence of the test results of performance-based critical thinking tests for undergraduate students

•The data were analysed using both the individual level and group level analysis.•A third of the students performed better on the MCQ than on the CLA.•One fourth performed better on the CLA than the MCQ.•Ten percent of students had completely opposite test results.•Thorough Processing and Superficial Processing groups were identified.

This article compares the test results of two different performance-based assessments of critical thinking: a constructed-response task from the Collegiate Learning Assessment (CLA) and a multiple-choice questionnaire (MCQ). These tests ostensibly measure the same critical thinking skills, such as analysing, interpreting and evaluating information and problem solving. The study utilised a mixed-method approach to explore the differences in students’ (n = 330) test scores. The results showed that the correspondence between the CLA and the MCQ was fully comparable in 45.5% of the students’ test performances. Ten percent of the students had completely opposite test results. Explanations for the inconsistent results are discussed in detail.