Bayesian predictive kernel discriminant analysis

• We combined Bayesian methods with kernel estimator to propose a new discriminant analysis.
• This new procedure can be adjusted if components of the feature vectors are not independent.
• This procedure is not computationally intensive.
• The new method proposed does not require knowledge of the data generating probabilistic model.
• The method is easy to implement and can be used in datasets of any dimension.

Discriminant analysis using Kernel Density Estimator (KDE) is a common tool for classification, but depends on the choice of the bandwidth or smoothing parameter of kernel. In this paper, we introduce a Bayesian Predictive Kernel Discriminant Analysis (BPKDA) eliminating this dependence by integrating the KDE with respect to an appropriate prior probability distribution for the bandwidth. Keypoints of the method are: (1) the formulation of the classification rule in terms of mixture predictive densities obtained by integrating kernel; (2) use of Independent Components Analysis (ICA) to choose a transform matrix so that transformed components are as independent as possible; and (3) nonparametric estimation of the predictive density by KDE for each independent component. Results on benchmark data sets and simulations show that the performance of BPKDA is competitive with, and in some cases significantly better than, Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA) and Naives Bayes discriminant Analysis with normal distribution (NNBDA).