Multi-Manifold based Rotation Forest for classification

Rotation Forest (RF) is a powerful ensemble classifier which has attracted substantial attention due to its performance. The RF algorithm uses Principal Component Analysis (PCA) for constructing the rotation matrix and extracting new features. In this paper, with the aim of extracting new features, three well-known manifold learning techniques are utilized to extract new features and incorporate into PCA for feature extraction. This new RF algorithm is hereby called Multi-Manifold RF (MMRF), and several experiments are conducted in the present study in order to evaluate its performance. The obtained results reported for nineteen datasets show the high efficiency of MMRF compared to fourteen state-of-the-art ensemble methods in terms of classification accuracy and computational effort. Furthermore, two statistical non-parametric tests (Friedman and Wilcoxon) are carried out to compare the average classification accuracies of MMRF with those of the other methods The experimental results demonstrate that MMRF outperforms twelve of these methods, while there is no significant difference between MMRF and the other two powerful ensemble-based methods, namely the SES-NSGAII and the IDES-P.