The formal relationship between analytic and bootstrap approaches to parametric inference

- Both bootstrap and analytic approximations yield Uniform(0,1) p-values to Op(n−3∕2).
- However, given a dataset, the p-values agree only to O(n−1).
- In practice, differences in inferences are small, but large simulations are required.

Two routes most commonly proposed for accurate inference on a scalar interest parameter in the presence of a (possibly high-dimensional) nuisance parameter are parametric simulation ('bootstrap') methods, and analytic procedures based on normal approximation to adjusted forms of the signed root likelihood ratio statistic. Under some null hypothesis of interest, both methods yield p-values which are uniformly distributed to error of third-order in the available sample size. But, given a specific dataset, what is the formal relationship between p-values calculated by the two approaches? We show that the two methodologies give the same inference to second order in general: the analytic p-value calculated from a dataset will agree with the bootstrap p-value constructed from that same dataset to O(n−1), where n is the sample size. In practice, the agreement is often startling.