The Inter-rater Reliability of Shoulder Arthroscopy

Purpose: The purpose of this study was to determine the inter-rater reliability of orthopaedic shoulder surgeons in evaluating the intra-articular structures involved in real-time diagnostic shoulder arthroscopy by use of a videotape model. Methods: Twenty patients (nineteen male patients and one female patient) diagnosed with recurrent anterior shoulder instability with a mean age of 27.9 years (range, 15 to 44 years) consented to participate in the study. Standardized diagnostic shoulder arthroscopies (modified Snyder protocol) were done by a single experienced surgeon, and the procedures were videotaped. The patients’ clinical information, radiographs, and videotape of the arthroscopy were sent to 6 experienced shoulder surgeons. The surgeons reviewed the clinical information, assessed the intra-articular structures shown on the videotape, and recorded their diagnoses on a standardized data collection form. The primary outcome was the median overall percent agreement for the video review surgeons by structure evaluated. Results: The median values for overall agreement for the video review surgeons were as follows: anterior labrum, 90%; inferior labrum, 75%; superior labrum, 60%; posterior labrum, 65%; superior glenohumeral ligament, 50%; middle glenohumeral ligament, 50%; anterior inferior glenohumeral ligament, 25%; glenoid surface, 35%; detection of Hill-Sachs lesion, 85%; biceps tendon, 70%; supraspinatus tendon, 85%; infraspinatus tendon, 70%; and subscapularis tendon, 80%. Conclusions: The inter-rater reliability for orthopaedic shoulder surgeons’ arthroscopic assessment of intra-articular anatomy in patients with anterior shoulder instability varied by structure examined. It was very good (>80%) for the anterior labrum and supraspinatus tendon and in detecting a Hill-Sachs lesion, poor (<40%) for the glenoid and anterior inferior glenohumeral ligament, and intermediate for all other structures examined. Level of Evidence: Level V, diagnostic study, expert opinion.