Empirical evaluation suggests Copas selection model preferable to trim-and-fill method for selection bias in meta-analysis

ObjectiveMeta-analysis yields a biased result if published studies represent a biased selection of the evidence. Copas proposed a selection model to assess the sensitivity of meta-analysis conclusions to possible selection bias. An alternative proposal is the trim-and-fill method. This article reports an empirical comparison of the two methods.Study Design and SettingWe took 157 meta-analyses with binary outcomes, analyzed each one using both methods, then performed an automated comparison of the results. We compared the treatment estimates, standard errors, associated P-values, and number of missing studies estimated by both methods.ResultsBoth methods give similar point estimates, but standard errors and P-values are systematically larger for the trim-and-fill method. Furthermore, P-values from the trim-and-fill method are typically larger than those from the usual random effects model when no selection bias is detected. By contrast, P-values from the Copas selection model and the usual random effects model are similar in this setting. The trim-and-fill method reports more missing studies than the Copas selection model, unless selection bias is detected when the position is reversed.ConclusionsThe assumption that the most extreme studies are missing leads to excessively conservative inference in practice for the trim-and-fill method. The Copas selection model appears to be the preferable approach.