The minimum coverage probability of confidence intervals in regression after a preliminary F test

Consider a linear regression model with regression parameter β=(β1,…,βp)β=(β1,…,βp) and independent normal errors. Suppose the parameter of interest is θ=aTβθ=aTβ, where aa is specified. Define the s  -dimensional parameter vector τ=CTβ−tτ=CTβ−t, where CC and tt are specified. Suppose that we carry out a preliminary F   test of the null hypothesis H0:τ=0H0:τ=0 against the alternative hypothesis H1:τ≠0H1:τ≠0. It is common statistical practice to then construct a confidence interval for θθ with nominal coverage 1−α1−α, using the same data, based on the assumption that the selected model had been given to us a priori   (as the true model). We call this the naive 1−α1−α confidence interval for θθ. This assumption is false and it may lead to this confidence interval having minimum coverage probability far below 1−α1−α, making it completely inadequate. We provide a new elegant method for computing the minimum coverage probability of this naive confidence interval, that works well irrespective of how large s is. A very important practical application of this method is to the analysis of covariance  . In this context, ττ can be defined so that H0 expresses the hypothesis of “parallelism”. Applied statisticians commonly recommend carrying out a preliminary F test of this hypothesis. We illustrate the application of our method with a real-life analysis of covariance data set and a preliminary F test for “parallelism”. We show that the naive 0.95 confidence interval has minimum coverage probability 0.0846, showing that it is completely inadequate.