Orthogonal One Step Greedy Procedure for heteroscedastic linear models

This paper investigates the prediction problem in the general Gaussian linear model with correlated noise, under the assumption that the covariance matrix is known, and focuses particularly on the high dimensional setting. We adapt an overly greedy procedure, where the relevant covariates are selected initially in one pass on the data, without any iteration, nor optimization. A simple componentwise regression, followed by an adaptive thresholding, locates leaders among the regressors to reduce the initial dimensionality. A second adaptive thresholding is performed on the linear regression upon the leaders. These steps take into account the correlated structure of the noise, by using weights associated to the covariates in a modified norm induced by the covariance matrix of the noise. The consistency of the procedure is investigated, and rates are provided for a wide range of sparsity classes, with little restriction on the number of regressors. An extensive computational experiment is conducted to emphasize the fact that the good theoretical results are corroborated by quite good practical performances in the presence of correlated noise.