Fully Bayesian analysis of the relevance vector machine with an extended hierarchical prior structure

This paper proposes an extended hierarchical hyperprior structure for kernel regression with the goal of solving the so-called Neyman–Scott problem inherent in the now very popular relevance vector machine (RVM). We conjecture that the proposed prior helps achieve consistent estimates of the quantities of interest, thereby overcoming a limitation of the original RVM for which the estimates of the quantities of interest are shown to be inconsistent. Unlike the majority of other authors in this area who typically use an empirical Bayes approach for RVM, we adopt a fully Bayesian approach. Our consistency claim at this stage remains only a conjecture, to be proved theoretically in a subsequent paper. However, we use a computational argument to demonstrate the merits of the proposed solution.