Comparison of classification methods with “n-class” receiver operating characteristic curves: A case study of energy drinks

• Four classification methods (PLS, LDA, RF, Boosted tree) were compared in a case study of energy drinks.
• Receiver operating characteristic (ROC) curves were calculated for the classification models.
• n-Class ROC curves were introduced as a new multiclass evaluation/visualization method.
• The method incorporates the Hanley formula and the weighted average of reduced, two-class problems.
• All methods classify the energy drinks successfully by their sugar content.

Four classification methods were compared using receiver operating characteristic (ROC) curves to identify the best one: two common ones (linear discriminant analysis, LDA and partial least squares, PLS) and another two (random forest, boosted tree) that are not applied as frequently as LDA or PLS yet. A dataset with 90 commercially available (in Hungary) energy drink samples were studied. Near-infrared (NIR) spectra were utilized for the classification of the energy drinks into three “natural” groups based on their sugar content. Another dataset, which contained the first ten principal components (PCs) was also used because of the limitation in the number of variables for LDA. The models were validated using n-fold cross-validation and randomization test. A new practice was elaborated to compare the pattern recognition methods with ROC curves. This new methodology was designed to provide an easy and straightforward way for the calculation of ROC curves for multi-class classification problems.In each case the energy drink samples could be classified to the appropriate groups very accurately. The best ROC curve belonged to the boosted tree method, but all of the studied methods were able to classify the samples to a great extent of correctness. The use of AUC values instead of correct classification rates can be a viable option for method comparison and also as a classification parameter.