Reliability of Speech-Language Pathologist and Otolaryngologist Ratings of Laryngeal Signs of Reflux in an Asymptomatic Population Using the Reflux Finding Score

SummaryObjective: To determine inter- and intrajudge agreement in rating signs of laryngopharyngeal reflux (LPR) under “ideal” conditions: Experienced coworkers in a practice devoted to voice-disordered patients, raters trained in the items on a standardized scale, raters from both speech-language pathology (SLP) and otolaryngology, and raters of asymptomatic participants. Study Design: Prospective study using a scale to rate videolaryngoscopic examinations. Methods: Two SLPs and two otolaryngologists used the Reflux Finding Scale (RFS) to independently rate videotapes of endoscopic examinations for 30 participants asymptomatic of reflux. Results: Thirteen (43%) were assigned a total score >7, indicative of LPR, by at least one rater. Intraclass correlation coefficients showed a significant lack of agreement in total scores provided by the otolaryngologists and by all raters combined. One otolaryngologist and the two SLPs demonstrated good interrater agreement in total scores. McNamar's statistic and Poisson regression modeling showed differences in rater agreement for many individual items. Repeated ratings of four participants showed no significant differences, indicating good intrarater reliability. Conclusions: Level of rater agreement regarding the presence and the severity of physical findings attributed to LPR within and between otolaryngologists and SLPs differed. Given the role each profession plays in the diagnosis and treatment of LPR and related voice disturbances, higher levels of interprofessional agreement are desired. Results support the need for greater consensus among professionals regarding the discreet features of physical findings associated with LPR, a fuller understanding of normal variants, and greater emphasis on interrater reliability when rating physical findings.