Multi-objective learning of Relevance Vector Machine classifiers with multi-resolution kernels

The Relevance Vector Machine (RVM) is a sparse classifier in which complexity is controlled with the Automatic Relevance Determination prior. However, sparsity is dependent on kernel choice and severe over-fitting can occur.We describe multi-objective evolutionary algorithms (MOEAs) which optimise RVMs allowing selection of the best operating true and false positive rates and complexity from the Pareto set of optimal trade-offs. We introduce several cross-validation methods for use during evolutionary optimisation. Comparisons on benchmark datasets using multi-resolution kernels show that the MOEAs can locate markedly sparser RVMs than the standard, with comparable accuracies.

► The RVM is a sparse classifier which uses ARD to control complexity.
► Kernel choice can lead to severe overfitting.
► Multi-objective evolutionary algorithms provide Pareto set of optimal trade-offs.
► Objectives are true positive rate, false positive rate and complexity.
► K-fold cross-validation during optimisation effectively controls over-fitting.