A probabilistic model for latent least squares regression

By far, least squares regression (LSR) is the most widely used data modeling method in statistics and mathematics because of its effectiveness and completeness. It plays an important underlying role in many extensions, e.g., regularized LSR, weighted LSR, and lasso. Since LSR is a discriminative model, it allows only sampling of the target variables conditioned on observations. In this paper, we present the latent LSR (LLSR), a generative model, which enables LSR to exploit the structural information hidden in the explanatory variables by imposing a sparsity-encouraging prior over the precision matrix of the latent variable. A maximum a posteriori (MAP) estimate is applied to obtain a point estimate of the model parameters. Both the toy example and real data tests suggest the effectiveness of LLSR.