Analysing data from observer studies in medical imaging research: An introductory guide to free-response techniques

Observer performance methods maintain their place in radiology research, particularly in the assessment of the diagnostic accuracy of new and existing techniques, despite not being fully embraced by the wider audience in medical imaging. The receiver operating characteristic (ROC) paradigm has been widely used in research and the latest location sensitive methods allow an analysis that is closer to the clinical scenario. This paper discusses the underpinning theories behind observer performance assessment, exploring the potential sources of error and the development of the ROC method. The paper progresses by explaining the clinical relevance and statistical suitability of the free-response ROC (FROC) paradigm, and the methodological considerations for those wishing to perform an observer performance study.