Should a Single Clinician Examiner be used in Objective Structure Clinical Examination?

The consistency in ratings provided by examiners can affect the reliability of the OSCE. This study was designed to identify the inter-rater reliability among examiners. This was a cross-sectional study involving 22 clinicians from all disciplines of medicine. An OSCE video was shown to each group and the examiners were to give their marks. The results showed a strong agreement among examiner in mixed discipline group (ICC=0.83), obstetrics & gynaecology group (ICC=0.84) and psychiatry group (ICC=0.88). Further analysis showed a moderate agreement among expert examiner (ICC=0.70) and non-expert examiners (ICC=0.78). We concluded that regardless the number of clinicians used as examiners in OSCE assessment, early exposure on OSCE, comprehensive training and OSCE station briefing prior to the examination are recommended.