Outcomes in systematic reviews of complex interventions never reached “high” GRADE ratings when compared with those of simple interventions

ObjectivesTo investigate the application of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach and the quality of evidence ratings in systematic reviews of complex interventions.Study Design and SettingThis study examined all 40 systematic reviews published in three Cochrane Review Groups from 2013 to May 2014: Cochrane Developmental, Psychosocial and Learning Problems Group (CDPLPG); Cochrane Public Health Group (CPHG); and Cochrane Depression, Anxiety, and Neurosis Group (CCDAN). The reviews were coded and classified into “complex” (n = 24) and “simple” (n = 16) intervention review groups based on the predefined complexity dimensions from the extant literature mapped into the PICOTS framework. All the data were analyzed in these two groups to help identify specific patterns of the GRADE ratings in the reviews of complex interventions.ResultsOutcomes of complex intervention reviews had higher proportions of “very low” quality of evidence ratings compared with those of simple intervention reviews (37.5% vs. 9.1% for the primary benefit outcomes) and were more frequently downgraded for inconsistency, performance bias, and study design. None of the outcomes of complex intervention reviews (0%) were given “high” GRADE ratings.ConclusionResults suggest that the GRADE assessment may not adequately describe the evidence base of complex interventions.