Is there more than one approach to evaluating the variability of surgeons' performance?

• There were statistically significant differences between seven surgeons' performances of shoulder surgery.
• However, only a non-significant part of this variability could be interpolated across the entire shoulder surgery field.
• Variability in performance within a group of surgeons cannot be generalized without additional statistical analyses.

IntroductionTo demonstrate that the variability found to be significant between surgeons' performances within a small group does not necessarily mean that this significance applies to the entire field of that specific type of surgery. It is common for inferences and recommendations for an entire field to be based on the variability within a small group of surgeons. The variability between groups usually remains unknown.MethodsAn analysis of variance was used to assess the statistical significance of the variability among surgeons' performances of a specific type of surgery within the studied sample. The intraclass correlation coefficient was used to investigate how large a segment of this variability can be explained by a surgeon-related factor for the entire surgeon population of a specific field. The topic was illustrated using data obtained from a group of seven surgeons who operated on the penetrating rotator cuff tears of 742 patients.ResultsThere were statistically significant differences between seven surgeons in the improvement of pain and the range of shoulder joint motion. However, only a small (≤2%) and statistically non-significant part of this variability could be explained by a difference between surgeons when the results were interpolated across the entire population of shoulder surgeons.Discussion and conclusionVariability in performance within a group of surgeons performing a specific type of surgery cannot be generalized to include the performance of all surgeons doing the same type of surgery without additional statistical analyses.