A pre-test like estimator dominating the least-squares method

We develop a pre-test type estimator of a deterministic parameter vector β in a linear Gaussian regression model. In contrast to conventional pre-test strategies, that do not dominate the least-squares (LS) method in terms of mean-squared error (MSE), our technique is shown to dominate LS when the effective dimension is greater than or equal to 4. Our estimator is based on a simple and intuitive approach in which we first determine the linear minimum MSE (MMSE) estimate that minimizes the MSE. Since the unknown vector β is deterministic, the MSE, and consequently the MMSE solution, will depend in general on β and therefore cannot be implemented. Instead, we propose applying the linear MMSE strategy with the LS substituted for the true value of β to obtain a new estimate. We then use the current estimate in conjunction with the linear MMSE solution to generate another estimate and continue iterating until convergence. As we show, the limit is a pre-test type method which is zero when the norm of the data is small, and is otherwise a non-linear shrinkage of LS.