Margin & diversity based ordering ensemble pruning

Ensemble pruning is a technique used to improve ensemble performance and reduce the ensemble size by selecting an optimal or sub-optimal subset as the final ensemble for prediction. In this research, using example margin and ensemble diversity, we prove that the ensemble pruning method should focus more on the following two factors: (1) examples with small absolute margin and (2) classifiers that correctly classify more examples and contribute larger diversity. Based on this principle, we propose a novel metric called the margin & diversity based measure (MDM) to explicitly evaluate the importance of individual classifiers. By incorporating ensemble members in a decreasing order based on the MDM, sub-ensembles are formed such that users can select the top T ensemble members for predictions. Compared to the original ensemble and other state-of-the-art ensemble pruning methods, the proposed method shows better performance in terms of accuracy.