Building confidence regions for the ROC surface

•We study a nonparametric estimate of the ROC surface.•We introduce several metrics over the ROC space.•We present and analyze a resampling procedure that build confident region.•We illustrate the accuracy of the method.

The ROC surface is the major criterion for assessing the accuracy of diagnosis test statistics s(X)s(X) in regard to their capacity of discriminating between K⩾3K⩾3 statistical populations. It provides additionally a widely used visual tool in the cases K=2K=2 and K=3K=3. It is the main purpose of this paper to investigate how to bootstrap a natural empirical estimator of the ROC surface in order to build accurate confidence regions in the ROC space. We first introduce a resampling procedure based on smooth versions of the empirical distributions involved to construct non Gaussian confidence regions. Simulation results are then displayed to show that such a “smoothed bootstrap” technique is preferable to a “naive” bootstrap approach in this situation. The accuracy of the method proposed is also illustrated using a psychometric dataset. An asymptotic analysis providing a rigorous theoretical basis for the method proposed is finally carried out in a functional framework.