Multiclass Classification with Cross Entropy-Support Vector Machines

In this paper, an important sampling method – Cross Entropy method is presented to deal with solving support vector machines (SVM) problem for multiclass classification cases. The use of this method is intended to accelarate the process of finding solution without sacrificing its quality. Using one-against-rest (OAR) and one-against-one (OAO) approaches, several binary SVM classifiers are constructed and combined to solve multiclass classification problems. For each binary SVM classifier, the cross entropy method is applied to solve dual SVM problem to find the optimal or at least near optimal solution, in the feature space through kernel map. For the meantime only RBF kernel function is investigated intensively. Experiments were done on four real world data sets. The results show one-against-rest produces better results than one-against-one in terms of computing time and generalization error. In addition, applying cross entropy method on multiclass SVM produces comparable results to the standard quadratic programming SVM in terms of generalization error.