Study-design selection criteria in systematic reviews of effectiveness of health systems interventions and reforms: A meta-review

At present, there exists no widely agreed upon set of study-design selection criteria for systematic reviews of health systems research, except for those proposed by the Cochrane Collaboration's Effective Practice and Organisation of Care (EPOC) review group (which comprises randomized controlled trials, controlled clinical trials, controlled before-after studies, and interrupted time series). We conducted a meta-review of the study-design selection criteria used in systematic reviews available in the McMaster University's Health Systems Evidence or the EPOC database. Of 414 systematic reviews, 13% did not indicate any study-design selection criteria. Of the 359 studies that described such criteria, 50% limited their synthesis to controlled trials and 68% to some or all of the designs defined by the EPOC criteria. Seven out of eight reviews identified at least one controlled trial that was relevant for the review topic. Seven percent of the reviews included either no or only one relevant primary study. Our meta-review reveals reviewers’ preferences for restricting synthesis to controlled experiments or study designs that comply with the EPOC criteria. We discuss the advantages and disadvantages of the current practices regarding study-design selection in systematic reviews of health systems research as well as alternative approaches.