Multiple test procedures using an upper bound of the number of true hypotheses and their use for evaluating high-dimensional EEG data

Frequency analyses of EEG data yield large data sets, which are high-dimensional and have to be evaluated statistically without a large number of false positive statements. There exist several methods to deal with this problem in multiple comparisons. Knowing the number of true hypotheses increases the power of some multiple test procedures, however the number of true hypotheses is unknown, in general, and must be estimated. In this paper, we derive two new multiple test procedures by using an upper bound for the number of true hypotheses. Our first procedure controls the generalized family-wise error rate, and thus is an improvement of the step-down procedure of Hommel and Hoffmann [Hommel G., Hoffmann T. Controlled uncertainty. In: Bauer P. Hommel G. Sonnemann E., editors. Multiple Hypotheses Testing, Heidelberg: Springer 1987;ISBN 3540505598:p. 154–61]. The second new procedure controls the false discovery proportion and improves upon the approach of Lehmann and Romano [Lehmann E.L., Romano J.P. Generalizations of the familywise error rate. Ann. Stat. 2005;33:1138–54]. By Monte-Carlo simulations, we show how the gain in power depends upon the accuracy of the estimate of the number of true hypotheses. The gain in power of our procedures is demonstrated in an example using EEG data on the processing of memorized lexical items.