Cost-effectiveness of classification ensembles

• Proposed accrual function compares two ensembles on the basis of performance and size.
• It returns a single numerical value as the result of comparison.
• It can be parameterized to bias towards smaller size/high performance ensembles.
• Its utility is demonstrated by applying the function to select published results.
• Applications with memory constraints can bias the choice to smaller ensembles.

Ensemble pruning is an important task in supervised learning because of the performance and efficiency advantage it begets to predictive modelling. Performance based empirical comparison (primarily on accuracy) is the most common modus operandi for critical evaluation of ensembles pruned by different algorithms. Deep analysis of existing literature reveals that ensemble size is an ignored attribute while judging the quality of ensembles.In this paper, we argue that the cost-effectiveness of an ensemble is a function of both performance and size. Hence, equitable comparison of two ensembles must take into account both these attributes to judge their relative merits. Following this argument, we propose an objective function called accrual function which quantifies the difference in performance and size of two ensembles, to gauge their relative cost-effectiveness. The function can be parameterized and has nice mathematical properties. Semantic interpretations of these properties are delineated in the paper. Finally, we apply the accrual function on published results from selected publications and demonstrate its ability to beget clarity while comparing ensembles.