A unified view of class-selection with probabilistic classifiers

• A unified framework of class-selection by the use of probabilistic equivalence is presented.
• A learning procedure for adapting the probabilistic equivalence is proposed.
• A normalized performance measure for class-selection is described.
• Experiments on real world datasets, using different classifiers, show the (statistical) significance of the proposition.

The possibility of selecting a subset of classes instead of one unique class for assignation is of great interest in many decision making systems. Selecting a subset of classes instead of singleton allows to reduce the error rate and to propose a reduced set to another classifier or an expert. This second step provides additional information, and therefore increases the quality of the result. In this paper, a unified view of the problem of class-selection with probabilistic classifiers is presented. The proposed framework, based on the evaluation of the probabilistic equivalence, allows to retrieve class-selective frameworks that have been proposed in the literature. We also describe an approach in which the decision rules are compared by the help of a normalized area under the error/selection curve. It allows to get a relative independence of the performance of a classifier without reject option, and thus a reliable class-selection decision rule evaluation. The power of this generic proposition is demonstrated by evaluating and comparing it to several state of the art methods on nine real world datasets, and four different probabilistic classifiers.