The impact of image test bank construction on radiographic interpretation outcomes: A comparison study

• High prevalence image test banks over-estimate the ability of observers.
• Clinical workload test banks may better reflect image interpretation competency.
• Clinical workload test banks can be applied across specialities and modalities.
• The model presented complements the traditional model for educational assessment.

IntroductionAssessment of image interpretation competency is commonly undertaken through review of a defined image test bank. Content of these image banks has been criticised for the high percentage of abnormal examinations which contrasts with lower reported incidences of abnormal radiographs in clinical practice. As a result, questions have been raised regarding the influence of prevalence bias on the accuracy of interpretive decision making. This article describes a new and novel approach to the design of musculoskeletal image test banks.MethodsThree manufactured image banks were compiled following a standard academic menu in keeping with previous studies. Three further image test banks were constructed to reflect local clinical workload within a single NHS Trust. Eighteen radiographers, blinded to the method of test bank composition, were randomly assigned 2 test banks to review (1 manufactured, 1 clinical workload). Comparison of interpretive accuracy was undertaken.ResultsInter-rater agreement was moderate to good for all image banks (manufactured: range k = 0.45–0.68; clinical workload: k = 0.49–0.62). A significant difference in mean radiographer sensitivity was noted between test bank designs (manufactured 87.1%; clinical workload 78.5%; p = 0.040, 95% CI = 0.4–16.8; t = 2.223). Relative parity in radiographer specificity and overall accuracy was observed.ConclusionThis study confirms the findings of previous research that high abnormality prevalence image banks over-estimate the ability of observers to identify abnormalities. Assessment of interpretive competency using an image bank that reflects local clinical practice is a better approach to accurately establish interpretive competency and the learning development needs of individual practitioners.