Probability estimation for multi-class classification using AdaBoost

• A probability estimation model is present for classification.
• The model reveals a new view of boosting.
• The model provides a new channel for constructing classifier structure.
• An analytical solution is given for pairwise coupling.
• A robust prediction framework is proposed for pairwise coupling.

It is a general viewpoint that AdaBoost classifier has excellent performance on classification problems but could not produce good probability estimations. In this paper we put forward a theoretical analysis of probability estimation model and present some verification experiments, which indicate that AdaBoost can be used for probability estimation. With the theory, we suggest some useful measures for using AdaBoost algorithms properly. And then we deduce a probability estimation model for multi-class classification by pairwise coupling. Unlike previous approximate methods, we provide an analytical solution instead of a special iterative procedure. Moreover, a new problem that how to get a robust prediction with classifier scores is proposed. Experiments show that the traditional predict framework, which chooses one with the highest score from all classes as the prediction, is not always good while our model performs well.