Meta-analyses of small numbers of trials often agree with longer-term results

ObjectiveMany systematic reviews include only a few studies. It is unclear whether recommendations based on these will be correct in the longer term; hence, this article explores whether meta-analyses give reliable results after only a few studies.Study Design and SettingCumulative meta-analysis of data from 65 meta-analyses from 18 Cochrane systematic reviews was carried out. Various measures of closeness to the pooled estimate from all trials after three and five trials were included. Changes during the accumulation of evidence were noted.ResultsThe 95% confidence interval included the final estimate in 72% of meta-analyses after three studies and in 83% after five studies. It took a median of four (interquartile range: 1.25–6) studies to get within 10% of the final point estimate. Agreement between the results at three and five studies and the final estimate was not predicted by the number of participants, the number of events, τ2, or I2. Estimates could still change substantially after many trials were included.ConclusionMany of the conclusions drawn from systematic reviews with small numbers of included studies will be correct in the long run, but it is not possible to predict which ones.