Enhancement of multi-class support vector machine construction from binary learners using generalization performance

We propose several new methods to enhance multi-class support vector machines (SVMs) by applying the generalization performance of binary classifiers as the core idea. This concept is applied to the existing algorithms, i.e., the Decision Directed Acyclic Graph (DDAG), the Adaptive Directed Acyclic Graph (ADAG), and Max Wins. Although there have been many previous attempts to use information such as the margin size and number of support vectors as the performance estimators for binary SVMs, this type of information may not accurately reflect the actual performance of the binary SVMs. We demonstrate that the generalization ability that is evaluated using a cross-validation mechanism is more suitable for directly extracting the actual performance of binary SVMs than the previous methods. Our methods are built around this performance measure, and each of them is crafted to overcome the weakness of the previous algorithms. The proposed methods include the Modified Reordering Adaptive Directed Acyclic Graph (MRADAG), Strong Elimination of the classifiers (SE), Weak Elimination of the classifiers (WE), and Voting-based Candidate Filtering (VCF). The experimental results demonstrate that our methods are more accurate than traditional methods. In particular, WE provides superior results compared to Max Wins, which is recognized as one of the most powerful techniques, in terms of both accuracy and classification speed with two times faster in average.