Video assessment of basic surgical trainees' operative skills

BackgroundWe previously described a feasible, reliable, valid, and acceptable clinical assessment form for basic surgical trainees (BSTs). We now recently assessed tissue-handling skills using real-time assessment (RTA) and video assessment (VA) and addressed feasibility, reliability, validity, and trainer–trainee agreement using the same assessment form.MethodsNine BSTs were videotaped as they performed open inguinal hernia repairs at 6 and 12 months of surgical training. Edited videotapes were independently rated by 7 consultants and 5 trainees using the technical part of the Edinburgh BST Assessment Form (EBSTAF-Tech), the Toronto Global Rating Scale of Surgical Performance (Toronto scale), and a visual analogue scale.ResultsRTA and VA both proved to be reliable tools (RTA α ≥ .85; VA α ≥ .76; interclass correlation coefficient [ICC] ≥ .69, rho ≥ .694 [P ≤ .004]), but RTA was not construct valid. VA scores distinguished consultant from trainee operators (Mann-Whitney P < .02), with trainers able to discriminate between trainee levels (Wilcoxon P = .01 to .023). Concurrent validity was demonstrated by the VA (trainers tau-b = .71 to .79 [P < .001] and trainees tau-b = .79 to .82 [P < .001]) with good trainer–trainee agreement (EBSTAF-Tech tau-b = .35 [P = .03), Toronto tau-b = .46 [P = .006), and visual analogue scale tau-b = .46 [P = .006]).ConclusionsVA of BST tissue-handling skills is feasible, reliable, valid, and highly sensitive. It may also improve trainee self-assessment skills by promoting reflective practice.