A filter for “confidence interval PP-values”

When a statistical test involves a nuisance parameter, an exact unconditional PP-value is found from a supremum search over the parameter space of the nuisance parameter. The result is less conservative than the corresponding (conditional) exact test. Restricting the region of the supremum search to a confidence interval gives a “confidence interval PP-value” which, after an appropriate adjustment, is also exact. We provide a filter to help identify cases for which this exact procedure reduces conservativeness even further, and we illustrate with numerical examples. These examples are also used to address questions about the optimum choice of confidence interval for the restricted supremum search, and to demonstrate the reduction in conservativeness attained by quasi-exact methods.