Kernel estimation for adjusted pp-values in multiple testing

Multiple testing procedures are frequently applied to biomedical and genomic research, for instance, identification of differentially expressed genes in microarray experiments. Resampling methods are commonly used to compute adjusted pp-values in multiple hypothesis testing problems. Importantly, the resampling-based multiple testing procedures are sensitive to the number of permutations, especially for the MinP adjustment procedure. The single-step MinP adjusted pp-values are derived from the distribution of the minimum of the pp-values. Because of computational complexity, the adjusted pp-values are often computed using the distribution of the maximum of the test statistics (MaxT). This paper proposes an approach based on the kernel density estimation (KDE) technique to reduce the number of permutations for implementing the single-step MinP adjustment. Simulation studies are conducted to demonstrate that the KDE method is more powerful than the MinP adjustment method under independent and correlated models. The three resampling-based single-step adjustment procedures, MaxT, MinP, and KDE, are applied to two published microarray data sets, the colon tumor data set consisting of 40 tumor and 22 normal colon tissue samples on 2000 human genes (endpoints) and the leukemia data set consisting of 27 acute lymphoblastic leukemia and 11 acute myeloid leukemia samples on 3051 genes. The MaxT adjusted pp-values are very robust to the number of permutations. The MaxT adjusted pp-values are stable with 10,000 permutations, while the MinP adjusted pp-values are step functions. As the number of permutations increases, the number of ties decrease. The adjusted pp-values are stable with 500,000 permutations. For the KDE method, the adjusted pp-values are stable at 50,000 permutations. At 1000,000 permutations, the three procedures have similar adjusted pp-values.