Probabilistic inference for multiple testing

•A probabilistic inference is developed for large-scale multiple testing.•A new multiple comparison procedure is obtained by the inferential model.•The method provides probabilities for the number of alternative hypotheses.•The method improves the Dempster–Shafer theory of belief.•The inference method is applied in gene expression data.

An inferential model is developed for large-scale simultaneous hypothesis testing. Starting with a simple hypothesis testing problem, the inferential model produces a probability triplet (p,q,r)(p,q,r) on an assertion of the null or alternative hypothesis. The probabilities p and q are for and against   the truth of the assertion, whereas r=1−p−qr=1−p−q is the remaining probability called the probability of “donʼt know”. For a large set of hypotheses, a sequence of assertions concerning the total number of true alternative hypotheses are proposed. The inferential model provides levels of belief without a prior for the sequence of assertions and offers a new multiple comparison procedure (MCP). The proposed method is obtained by improving Fisherʼs fiducial and the Dempster–Shafer theory of belief functions so that it produces probabilistic inferential results with desirable frequency properties. The new multiple comparison procedure is shown to have a comparable performance with other existing MCPs and is favorable in terms of probabilistic interpretation. The proposed method is applied in identifying differentially expressed genes in microarray data analysis.