The script concordance test as a measure of clinical reasoning: a national validation study

IntroductionThe script concordance test (SCT) is an innovative tool for clinical reasoning assessment. It has previously been shown to be a reliable and valid measure of clinical reasoning among general surgical residents.PurposeTo determine if the SCT maintained its validity and reliability when administered on a national level.MethodsThe test was administered to 202 residents (51 R1, 45 R2, 45 R3, 28 R4, and 33 R5) in 9 general surgery programs across Canada.ResultsThe optimized version of the test had a reliability (Cronbach alpha) of .85. Scores increased progressively from R1 (64.5 ± 7.6) to R2 (69.5 ± 5.8) to R3 (69.9 ± 6.7) to R4 (72.0 ± 6.2) with a dip in the R5s (68.3 ± 8.6). The test was able to differentiate junior (R1+ R2 = 66.8 ± 7.2) from senior residents (R3 + R4 + R5 = 70.0 ± 7.3, P = .001) across all the programs.ConclusionsThe SCT maintained its reliability and validity as a measure of intraoperative clinical reasoning among general surgical residents when administered across multiple centers. We believe that the SCT can be developed to measure clinical reasoning in high-stakes national examinations.