Intraobserver variability in grading severity of repeated identical cases of mitral regurgitation

BackgroundIn clinical practice, mitral regurgitation (MR) is often assessed from the visual impression of the color Doppler image. It is recognized that repeated scanning can give different images and that different observers may grade the same images differently. In this study, we focus on a more intrinsic source of variability—intraobserver variability in grading of identical images, presented more than once at the same sitting. Furthermore, we look for evidence that observer grading is influenced by the severity of the immediately preceding case viewed.MethodsAnonymous, 4-chamber color Doppler 2-dimensional video clips of MR were obtained from 60 unselected patients with MR. Six clips were identified by 2 observers as of contentious severity (either between mild and moderate or between moderate and severe). A 72-clip sequence was constructed from the 54 “uncontentious” selected cases of MR intermingled with 3 presentations of each of the 6 “contentious” images. Each contentious image was shown once without a designed order, once preceded by 3 clips of less severe MR, and once preceded by 3 clips of more severe MR.ResultsOnly 1 (8%) of 12 observers gave consistent gradings for the triply presented images. More than 90% (11/12) of the observers reported a different grading of the same clip of MR in at least 1 of the 6 cases. The MR severity was changed in 51.4% of the triplets of identical images shown. Of 12 reporters, 5 (42%) showed classification variability between severe and nonsevere grades in at least 1 of the 6 cases. The direction of change showed no sign of consistency between observers (P = .375).ConclusionEven a skilled observer cannot be relied upon to give an identical grading to an identical simple video clip of MR, when re-presented surreptitiously within a few minutes. Interobserver variability cannot therefore simply be blamed on differential levels of skill. Because, even under these ideal and dramatically simplified conditions, visual assessment is so variable, the future emphasis of echocardiographic MR grading may lie in integrating qualitative analysis with simple quantification methods.