FAMCDM: A fusion approach of MCDM methods to rank multiclass classification algorithms

Various methods and algorithms have been developed for multiclass classification problems in recent years. How to select an effective algorithm for a multiclass classification task is an important yet difficult issue. Since the multiclass algorithm selection normally involves more than one criterion, such as accuracy and computation time, the selection process can be modeled as a multiple criteria decision making (MCDM) problem. While the evaluations of algorithms provided by different MCDM methods are in agreement sometimes, there are situations where MCDM methods generate very different results. To resolve this disagreement and help decision makers pick the most suitable classifier(s), this paper proposes a fusion approach to produce a weighted compatible MCDM ranking of multiclass classification algorithms. Several multiclass datasets from different domains are used in the experimental study to test the proposed fusion approach. The results prove that MCDM methods are useful tools for evaluating multiclass classification algorithms and the fusion approach is capable of identifying a compromised solution when different MCDM methods generate conflicting rankings.

Research Highlights► Select effective algorithm for multiclass classification problems. ► Selection process modeled as a multiple criteria decision making (MCDM) problem. ► MCDM methods to evaluate 7 classification algorithms in 4 public domain datasets. ► Fusion approach to produce a weighted compatible MCDM ranking. ► Identifying a compromised solution when conflicting rankings exist.