Surgical EducationFaculty evaluations of resident medical knowledge: can they be used to predict American Board of Surgery In-Training Examination performance?

•We examine evaluation tools at 2 general surgery residency programs.•Faculty evaluations do not correlate well with one another.•Faculty evaluations of medical knowledge correlate weakly with ABSITE scores.

BackgroundThe American Board of Surgery In-Training Examination (ABSITE) offers annual assessment of resident medical knowledge. We sought to determine if ongoing end-of-rotation evaluations by faculty of residents' medical knowledge correlate with ABSITE performance.MethodsRetrospective cross-sectional study was conducted over 3 years at 2 institutions. Faculty rated residents' clinical knowledge as part of a global summative evaluation. The intraclass correlation coefficient and convergent validity between faculty evaluations and ABSITE performance were assessed.ResultsA total of 1,562 faculty evaluations were completed for about 147 residents. There was poor agreement among faculty for each resident, with intraclass correlation coefficients of less than 0.2. Spearman's correlation coefficient was calculated for evaluations and ABSITE scores and were found to be weakly correlative at one institution and not correlated at all at the other. Finally, evaluations across quartiles of resident ABSITE scores were examined and show no correlation.ConclusionFaculty evaluations of resident medical knowledge correlate poorly with resident ABSITE performance, and should not be used as an ongoing predictive tool.