Direct and indirect tests for publication bias: asymmetry and sexual selection

Recently, several studies have suggested that the biological literature may be biased because studies with particular effect sizes are more likely to get published. However, only one published direct test comparing effect sizes of a representative sample of published and unpublished studies exists in biology. We performed an additional direct test of publication bias, using studies of the relationship between asymmetry and sexual selection. Samples of published and unpublished papers on this relationship were obtained by (1) checking the literature; (2) contacting scientists working on the subject; and (3) posting requests for studies against a reward on Web sites. There was no significant bias in respondents to our request as compared to nonrespondents with respect to effect size or other features of the published studies on the subject. Also, the characteristics of the studies of respondents with and without unpublished research on the subject did not differ significantly. Unpublished and published studies did not differ significantly in effect size adjusted for sample size, using random-effects meta-analysis. The number of unpublished studies we obtained, however, was moderate. Moreover, there was no evidence of a temporal change in mean effect size adjusted for sampling effort between a sample of studies conducted through 1996, a sample of studies conducted thereafter until June 2001, and the sample of unpublished studies in the present study. Indirect tests for publication bias used on the three samples revealed little evidence of publication bias. In conclusion, there is a small to intermediate effect size weighted by sample size of the relationship between asymmetry and sexual selection, and this effect appears to be largely independent of publication bias as revealed by direct and indirect estimates of publication bias.