Statistical methods can be improved within Cochrane pregnancy and childbirth reviews

ObjectivesTo assess statistical methods within systematic reviews of the Cochrane Pregnancy and Childbirth Group (CPCG).Study Design and SettingWe extracted details about statistical methods within 75 reviews containing at least 10 studies.ResultsThe median number of forest plots per review was 52 (min = 5; max = 409). Seven of the 75 reviews assessed publication bias or explained why not. Forty-four of the 75 reviews performed random-effects meta-analyses; just 1 of these justified the approach clinically and none interpreted its pooled result correctly. Of 31 reviews not using random-effects, 26 assumed a fixed-effect given potentially moderate or large heterogeneity (I2 > 25%). In their Methods section, 25 (33%) of the 75 reviews said I2 was used to decide between fixed-effect and random-effects; however, in 12 of these (48%) reviews, this was not carried out in their Results section. Of 72 reviews with moderate or large heterogeneity, 47 (65%) did not explore the causes of heterogeneity or justify why not.ConclusionWithin CPCG reviews, publication bias is rarely addressed; heterogeneity is often not appropriately considered, and random-effects analyses are incorrectly interpreted. How these shortcomings impact existing review conclusions needs further investigation, but regardless of this, we recomment the Cochrane Collaboration increase “hands-on” statistical support.