General Surgery Residents can Be a Reliable Resource in the Evaluation of Residency Applications

ObjectiveSurgical residents’ ability to screen general surgery (GS) applicants has not been previously investigated. The objective of this study was to compare surgical residents’ evaluation of Electronic Residency Application Service (ERAS) applicants to that of faculty using a standardized assessment instrument.DesignA prospective analysis of ERAS applications using a standardized assessment tool.SettingA university-affiliated, academic, county GS residency program.ParticipantsBefore the interview day, 51 ERAS (2013-2014) applications were reviewed by 10 different assessors (6 GS faculty, including the program director, and 4 GS residents), who evaluated applicants on 10 characteristics (subjective and objective) using a 5-point Likert scale, a total score, and a Global Rating Scale that ranked candidates into deciles.ResultsThere were a total of 510 assessments. In 8 of 10 individual domains the interrater reliability (IRR) between residents and faculty was good. The IRRs of the total score and global score were excellent. The Spearman ρ between the total score and final rank list were similar for faculty (−0.558) and residents (−0.592).ConclusionsThe excellent IRR score between the total and global scores of faculty and residents demonstrates the reliability of GS residents in evaluating ERAS applications. The low correlations between the total score and final rank are consistent with those in previous studies, in which the interview has been demonstrated to be the most important factor in determining final selection.