A New Instrument for Assessing Resident Competence in Surgical Clinic: The Ottawa Clinic Assessment Tool

BackgroundThe shift toward competency-based medical education has created a demand for feasible workplace-based assessment tools. Perhaps, more important than competence to assess an individual patient is the ability to successfully manage a surgical clinic. Trainee performance in clinic is a critical component of learning to manage a surgical practice, yet no assessment tool currently exists to assess daily performance in outpatient clinics for surgery residents. The development of a competency-based assessment tool, the Ottawa Clinic Assessment Tool (OCAT), is described here to address this gap.Study DesignA consensus group of experts was gathered to generate dimensions of performance reflective of a competent “generalist” surgeon in clinic. A 6-month pilot study of the OCAT was conducted in orthopedics, general surgery, and obstetrics and gynecology with quantitative and qualitative evidence of validity collected. In all, 2 subsequent feedback sessions and a survey for staff and residents evaluated the OCAT for clarity and utility.ResultsThe OCAT is a 9-item tool, with a global assessment item and 2 short-answer questions. Among the 2 divisions, 44 staff surgeons completed 132 OCAT assessments of 79 residents. Psychometric data was collected as evidence of validity. Analysis of feedback indicated that the entrustability rating scale was useful for surgeons and residents and that the items could be correlated with individual competencies.ConclusionsMultiple sources of validity evidence collected in this study demonstrate that the OCAT can measure resident clinic competency in a valid and feasible manner.