A better Beta for the H measure of classification performance

• We propose a modified standard distribution for the H measure in the case of unbalanced datasets.
• We emphasize the importance of the H measure as a coherent alternative to the AUC.
• We introduce severity ratios to facilitate problem-specific application of the H measure.

The area under the ROC curve is widely used as a measure of performance of classification rules. However, it has recently been shown that the measure is fundamentally incoherent, in the sense that it treats the relative severities of misclassifications differently when different classifiers are used. The H measure overcomes this by allowing a researcher to fix the distribution of relative severities to a classifier-independent setting on a given problem. This note extends the discussion, and proposes a modified standard distribution for the H   measure, which better matches the requirements of researchers, the Beta(π1+1,π0+1)Beta(π1+1,π0+1) distribution.