Bayesian high-dimensional screening via MCMC

• We propose an MCMC algorithm for Bayesian model selection in high-dimensional linear models.
• We investigate the theoretical property of the proposed Bayesian algorithm.
• The proposed Bayesian algorithm performs better than the existing methods.

We explore the theoretical and numerical properties of a fully Bayesian model selection method in the context of sparse high-dimensional settings, i.e., p≫np≫n, where pp is the number of covariates and nn is the sample size. Our method consists of (1) a hierarchical Bayesian model with a novel prior placed over the model space which includes a hyperparameter tntn controlling the model size and (2) an efficient MCMC algorithm for automatic and stochastic search of the models. Our theory shows that, when specifying tntn correctly, the proposed method yields selection consistency, i.e., the posterior probability of the true model asymptotically approaches one; when tntn is misspecified, the selected model is still asymptotically nested in the true model. The theory also reveals insensitivity of the selection result with respect to the choice of tntn. In implementations, a reasonable prior is further assumed on tntn. Our approach conducts selection, estimation and even inference in a unified framework. No additional prescreening or dimension reduction step is needed. Two novel gg-priors are proposed to make our approach more flexible. The numerical advantages of the proposed approach are demonstrated through comparisons with sure independence screening (SIS).