On control of the false discovery rate under no assumption of dependency

Most false discovery rate (FDR) controlling procedures require certain assumptions on the joint distribution of p-values. Benjamini and Hochberg [1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57, 289-300] proposed a step-up procedure with critical constants αi=(i/m)α,1⩽i⩽m, for a given level 0<α<1 and showed that FDR⩽(m0/m)α under the assumption of independence of p-values, where m is the total number of null hypotheses and m0 the number of true null hypotheses. Benjamini and Yekutieli [2001. The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29, 1165-1188] showed that for the same procedure FDR⩽(m0/m)α∑j=1m1/j, whatever may be the joint distribution of p-values. In one of the results in this paper, we show that this upper bound for FDR cannot be improved in the sense that there exists a joint distribution of p-values for which the upper bound is attained. A major thrust of this paper is to work in the realm of step-down procedures without imposing any condition on the joint distribution of the underlying p-values. As a starting point, we give an explicit expression for FDR specially tailored for step-down procedures. Using the same critical constants as those of the Benjamini-Hochberg procedure, we present a new step-down procedure for which the upper bound for FDR is much lower than what is given by Benjamini and Yekutieli. The explicit expression given for FDR and some optimization techniques stemming from the knapsack problem are instrumental in getting the main result. We also present some general results on stepwise procedures built on non-decreasing sequences of critical constants.