Scaling up twin support vector regression with safe screening rule

Twin support vector regression (TSVR) is a popular and efficient regression method, since it solves a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one as in the traditional SVR. However, it is time-consuming to deal with the large-scale problems, especially for the multi-parameter case. Inspired by the sparsity of TSVR, we propose an efficient safe screening rule based on variational inequality (VI) to accelerate TSVR, termed as SSR-TSVR. Through this rule, most 0 and 1 components in dual solution can be identified before actually training TSVR. Then the scale of the model will be extremely reduced by preassigning the identified components. In this way, the computational time of TSVR can be sharply shortened. There are two main advantages of our method: (1) it is safe in the sense that it guarantees to achieve the exactly same solution as solving original problem; (2) it is efficient for both the linear and nonlinear cases. Another contribution is that the dual coordinate descent method (DCDM) is employed to further accelerate the computational speed. Experimental results on twelve benchmark datasets demonstrate the efficiency and safety of our proposed method.