Building confidence regions for the ROC surface

• We study a nonparametric estimate of the ROC surface.
• We introduce several metrics over the ROC space.
• We present and analyze a resampling procedure that build confident region.
• We illustrate the accuracy of the method.

The ROC surface is the major criterion for assessing the accuracy of diagnosis test statistics s(X)s(X) in regard to their capacity of discriminating between K⩾3K⩾3 statistical populations. It provides additionally a widely used visual tool in the cases K=2K=2 and K=3K=3. It is the main purpose of this paper to investigate how to bootstrap a natural empirical estimator of the ROC surface in order to build accurate confidence regions in the ROC space. We first introduce a resampling procedure based on smooth versions of the empirical distributions involved to construct non Gaussian confidence regions. Simulation results are then displayed to show that such a “smoothed bootstrap” technique is preferable to a “naive” bootstrap approach in this situation. The accuracy of the method proposed is also illustrated using a psychometric dataset. An asymptotic analysis providing a rigorous theoretical basis for the method proposed is finally carried out in a functional framework.