Quadratic programming for class ordering in rule induction

• Rule learners solve a K class problem by converting it into 2-class problems.
• As heuristic, the classes are fed to the learner in the order of prior probabilities.
• In this paper, we propose a novel algorithm to improve this heuristic.
• The algorithm transforms the ordering search problem into a QP problem.
• Using the solution of the QP problem, algorithm extracts the optimal ordering.

Separate-and-conquer type rule induction algorithms such as Ripper, solve a K > 2 class problem by converting it into a sequence of K − 1 two-class problems. As a usual heuristic, the classes are fed into the algorithm in the order of increasing prior probabilities. Although the heuristic works well in practice, there is much room for improvement. In this paper, we propose a novel approach to improve this heuristic. The approach transforms the ordering search problem into a quadratic optimization problem and uses the solution of the optimization problem to extract the optimal ordering. We compared new Ripper (guided by the ordering found with our approach) with original Ripper (guided by the heuristic ordering) on 27 datasets. Simulation results show that our approach produces rulesets that are significantly better than those produced by the original Ripper.