The consistency test does not-and cannot-deliver what is advertised: A comment on Francis (2013)

The statistical consistency test of  Ioannidis and Trikalinos (2007) has been used recently by  Francis, 2012a, Francis, 2012c, Francis, 2012d, Francis, 2012e, Francis, 2013, Francis, in press, to argue that specific sets of experiments show evidence of publication bias. I argue that the test is unnecessary because publication bias exists almost everywhere as property of the research process, not individual studies. Furthermore, for several reasons, the test does not support the claims made on its behalf. Instead of focusing on testing sets of experiments for publication bias, we should focus on changes to scientific culture to reduce the bias.