Parsimonious Mahalanobis kernel for the classification of high dimensional data

The classification of high dimensional data with kernel methods is considered in this paper. Exploiting the emptiness property of high dimensional spaces, a kernel based on the Mahalanobis distance is proposed. The computation of the Mahalanobis distance requires the inversion of a covariance matrix. In high dimensional spaces, the estimated covariance matrix is ill-conditioned and its inversion is unstable or impossible. Using a parsimonious statistical model, namely the High Dimensional Discriminant Analysis model, the specific signal and noise subspaces are estimated for each considered class making the inverse of the class specific covariance matrix explicit and stable, leading to the definition of a parsimonious Mahalanobis kernel. A SVM based framework is used for selecting the hyperparameters of the parsimonious Mahalanobis kernel by optimizing the so-called radius-margin bound. Experimental results on three high dimensional data sets show that the proposed kernel is suitable for classifying high dimensional data, providing better classification accuracies than the conventional Gaussian kernel.

► A kernel function for the classification of high dimensional data is proposed.
► Data are modeled with a parsimonious statistical model.
► SVM is used with a optimization procedure for the estimation of the kernel hyperparameters.
► Results on several high dimension data have shown that the kernel performs better than conventional kernel.