Estimating the area under a receiver operating characteristic (ROC) curve: Parametric and nonparametric ways

Receiver operating characteristic (ROC) analysis has become a standard tool to tackle the two-sample problems in many scientific and engineering fields. The area under the curve (AUC) plays a leading role as a figure of merit to characterize the performances of diagnostic systems in medicine, binary classifiers in machine learning, and energy detectors in signal processing. Aiming at addressing some open problems of estimating the AUC, in this paper we deal with the AUC estimation problems in both parametric and nonparametric ways based on the equivalence between the AUC and Mann–Whitney U statistic (MWUS). In parametric ways, we derive the exact analytical expressions of the mean and variance of AUC for samples drawn from some important distributions that are frequently encountered in signal processing; whereas in nonparametric ways, we develop a rank-based algorithm which can estimate the variance of AUC unbiasedly and speedily. Monte Carlo simulations verify both our theoretical and algorithmic findings in this work.

• A general framework for AUC analysis is presented.
• Exact mean and variance of AUC are derived under important distributions.
• An efficient and unbiased algorithm is proposed for estimating the variance of AUC.