Brief critical review: Statistical assessment of biomarker performance

IntroductionArticles have suggested that C-statistics associated with receiver operator characteristic (ROC) curves are insufficiently sensitive compared to odds/risk ratios (OR/RR) defined by p-values. Some suggest reclassification techniques are more appropriate.MethodsWe review the concepts of p-values, OR/RR, and ROC curves. We construct a ROC curve, demonstrate parametric and nonparametric curves, and analyze a comparison of ROC curves.ResultsUsing these illustrations, we show that the ROC curve is not simply a C-statistic but a continuum of sensitivity/specificity pairs over decision levels. We demonstrate that p-values provide little useful information about discrimination and that OR/RR is a limited expression of a sensitivity/specificity plot. We illustrate that low prevalence produces low positive predictive values, reclassification techniques are subject to the same rules of prevalence as other analysis, and that modifying the analysis can decrease the p-value comparing C-statistics without altering the sensitivity/specificity plot.ConclusionsROC curves provide both visual and statistical information to support entry into large-scale trials, determining decision levels and the use of testing in the absence of such trials. If the sensitivity/specificity plot shows little improvement at appropriate decision point(s), statistically significant-improvement in other “novel” statistical indices should be suspect.

► ROC curves reflect more than just an area. ► ROC curves are a sensitivity/specificity plot of all data. ► Each point represents a decision cutoff. ► OR/RR reflect only a single point on the ROC curve. ► ROC curves provide more information to support biomarker effectiveness.