Article ID | Journal | Published Year | Pages | File Type |
---|---|---|---|---|
4947222 | Neurocomputing | 2017 | 33 Pages |
Abstract
Here, we propose Bayesian masking (BM) in order to resolve the trade-off problem between sparsity and shrinkage. Our strategy is not to directly impose any regularization on the weights; instead, BM introduces binary latent variables, called masking variables, into a regression model to keep the sparsity; each feature and sample has a binary variable whose value determines if the feature is masked or not at the sample. We derive a variational Bayesian inference algorithm for the augmented model based on the factorized information criterion (FIC), a recently-proposed asymptotic approximation of the marginal log-likelihood. We analyze the one-dimensional estimators of Lasso, automatic relevance determination (ARD), and BM, and thus show the superiority of BM in terms of the sparsity-shrinkage trade-off. Finally, we confirm our theoretical analyses through experiments and, demonstrate that BM achieves higher feature selection accuracy compared with Lasso and ARD.
Related Topics
Physical Sciences and Engineering
Computer Science
Artificial Intelligence
Authors
Yohei Kondo, Kohei Hayashi, Shin-ichi Maeda,