A probability-mapping algorithm for calibrating the posterior probabilities: A direct marketing application

Calibration refers to the adjustment of the posterior probabilities output by a classification algorithm towards the true prior probability distribution of the target classes. This adjustment is necessary to account for the difference in prior distributions between the training set and the test set. This article proposes a new calibration method, called the probability-mapping approach. Two types of mapping are proposed: linear and non-linear probability mapping. These new calibration techniques are applied to 9 real-life direct marketing datasets. The newly-proposed techniques are compared with the original, non-calibrated posterior probabilities and the adjusted posterior probabilities obtained using the rescaling algorithm of Saerens et al. (2002). The results recommend that marketing researchers must calibrate the posterior probabilities obtained from the classifier. Moreover, it is shown that using a ‘simple’ rescaling algorithm is not a first and workable solution, because the results suggest applying the newly-proposed non-linear probability-mapping approach for best calibration performance.

► Marketing researchers must calibrate the posterior probabilities obtained from the classifier.
► A ‘simple’ rescaling algorithm like Saerens et al. (2002) is not a first and workable calibration solution.
► Non-linear probability-mapping approaches deliver better calibration performance.