An efficient regularized K-nearest neighbor based weighted twin support vector regression

In general, pattern classification and regression tasks do not take into consideration the variation in the importance of the training samples. For twin support vector regression (TSVR), this implies that all the training samples play the same role on the bound functions. However, the number of close neighboring samples near to each training sample has an effect on the bound functions. In this paper, we formulate a regularized version of the KNN-based weighted twin support vector regression (KNNWTSVR) called RKNNWTSVR which is both efficient and effective. By introducing the regularization term and replacing 2-norm of slack variables instead of 1-norm, our RKNNWTSVR only needs to solve a simple system of linear equations with low computational cost, and at the same time, it improves the generalization performance. Particularly, we compare four implementations of RKNNWTSVR with existing approaches. Experimental results on several synthetic and benchmark datasets indicate that, comparing to SVR, WSVR, TSVR and KNNWTSVR, our proposed RKNNWTSVR has better generalization ability and requires less computational time.