Assessment of Procedural Skills Using Virtual Simulation Remains a Challenge

ObjectiveThe LAP Mentor is a procedural simulator that provides a stepwise training for laparoscopic cholecystectomy. This study addresses its “construct” validity that is present when a simulator is able to discriminate between persons with known differences in performance level on the laparoscopic cholecystectomy in real life.DesignThree groups with different skill levels performed 2 trials of 4 distinct parts of the cholecystectomy procedure (cholecystectomy exercises) and 1 full procedure on the LAP Mentor. Assessment parameters concerning the quantity and the quality of performance were compared between groups using the Kruskal-Wallis and Mann-Whitney U tests.SettingThe entire research was performed in the Center for Surgical Technologies, Leuven, Belgium.ParticipantsFor study purposes, 5 expert abdominal laparoscopists (>100 laparoscopic cholecystectomies performed), 11 surgical residents (10-30 cholecystectomies performed), and 10 novices (minimal laparoscopic experience) were recruited.ResultsWith regard to the quantity of performance (time needed and number of movements), the experts showed significantly better results compared with the novices in the cholecystectomy exercises. Only in the full procedure, the results of all the parameters (except speed) were significantly different between the 3 groups, with the best results observed for the experts and worst for the novices. With respect to quality of performance, only the parameter “accuracy rate of dissection” in exercise 3 showed significantly better performance by the experts.ConclusionsOnly the full procedure of the LAP Mentor procedural simulator has enough discriminative power to claim construct validity. However, the lack of quality control, which is indispensible in the evaluation of procedural skills, makes it currently unsuited for the assessment of procedural laparoscopic skills. The role of the simulator in a training context remains to be elucidated.