Assessment of undergraduate physiotherapist clinical performance: investigation of educator inter-rater reliability

ObjectivesTo investigate the inter-rater reliability of physiotherapy educators in awarding clinical placement marks to undergraduate students using a standardised clinical placement assessment form.DesignInter-rater reliability study performed over five undergraduate physiotherapy clinical placements.SettingFive clinical sites associated with a BSc undergraduate physiotherapy programme.ParticipantsSecond and final year physiotherapy students who were on clinical placements (n = 86 paired assessments). Two physiotherapy educators (a practice tutor and a practice educator), equally involved in supervising the students over the placement period, marked each student blindly at the end of the placement.Outcome measureMarking was performed using a standardised clinical placement assessment form and guidelines. This form was developed by physiotherapy educators, and utilised previously published work in the area.ResultsEighty-six paired assessment marks were analysed. Practice educators and practice tutors agreed on grades on 74% of occasions. The mean difference in marks (maximum of 100) between educators was −0.5 (95% confidence interval −1.1 to 0.2), and using the limits of agreement method, the results suggested that where two raters mark a student at the end of a clinical placement, the raters will be within 6.2 marks of each other on 95% of occasions. The intraclass correlation coefficient (ICC) for the overall mark was 0.84, indicating almost perfect agreement. Subsections of the form also had substantial agreement (patient management ICC, 0.75; professional development ICC, 0.75; organisation and management ICC, 0.81).ConclusionPhysiotherapy educators demonstrated a high level of reliability in the assessment and marking of undergraduate physiotherapy student performance using a standardised clinical assessment form. This was evident over several sites and specialities.