ROC curves for regression

• We present a natural and powerful counterpart of ROC analysis for regression.
• The parallelism with classification evaluation suggests high expected applicability.
• RROC space and RROC curves are derived from asymmetric loss functions.
• The area over the RROC curve is shown to be equivalent to the error variance.
• Predictive model behaviour will be better understood by these links (MSE, Bias, Var).

Receiver Operating Characteristic (ROC) analysis is one of the most popular tools for the visual assessment and understanding of classifier performance. In this paper we present a new representation of regression models in the so-called regression ROC (RROC) space. The basic idea is to represent over-estimation against under-estimation. The curves are just drawn by adjusting a shift, a constant that is added (or subtracted) to the predictions, and plays a similar role as a threshold in classification. From here, we develop the notions of optimal operating condition, convexity, dominance, and explore several evaluation metrics that can be shown graphically, such as the area over the RROC curve (AOC). In particular, we show a novel and significant result: the AOC is equivalent to the error variance. We illustrate the application of RROC curves to resource estimation, namely the estimation of software project effort.