A sparse method for least squares twin support vector regression

Recently, some nonparallel plane regressors, such as twin support vector regression (TSVR), and least squares TSVR (LSTSVR), have been proposed and have attracted much attention. However, these algorithms are not sparse, which would make their learning speed low. In this paper, we propose a novel nonparallel plane regressor, which can automatically select the relevant features. Firstly, we introduce a regularization term to the objective function of LSTSVR, which can guarantee two quadratic programming problems (QPPs) are strong convex, implying that the proposed algorithm can obtain the global but unique solution. Secondly, the primal formulation is converted to a linear programming (LP) problem. Then, we solve the dual of the LP formulation by minimizing its exterior penalty problem, which would make our method yield very sparse solutions. In other words, this method can suppress input features so that it can obtain comparable regression performance when using fewer computational time. Numerical experiments on artificial dataset and benchmark datasets demonstrate the feasibility and validity of the proposed algorithm.