A study on regularized Weighted Least Square Support Vector Classifier

Least Square Support Vector Machine(LSSVM) has been widely used for solving regression and classification problems due to its simple solution. However, LSSVM has some drawbacks in practice, such as lack of robustness, loss of sparseness of its solution and inapplicability of solving classification problem in some cases. In order to find robust and sparse solution for classification problem by using Least Square methods, we propose a novel regularized Weighted Least Square Support Vector Classifier in this paper. First, we give a basic optimized Weighted Least Square Support Vector Classifier model, which can be used to obtain the best weights according to the distance between samples and classification boundary and obtain an extremely sparse solution. In order to control the sparsity of solution, we further propose a regularized Weighted Least Square Support Vector Classifier model. After theoretical analysis of regularization function, we construct 2 kinds of regularization functions which meet our requirements and then design relative algorithms for optimizing weights and finding the best solution. The proposed method is evaluated on artificial datasets and several important benchmark databases in Machine Learning, and achieves the more encouraging results compared with some state-of-the-art approaches.