• TUCS Publication Series
• How to Publish in TUCS Series
• Publication Search
• Publication Graph
• Publication Input
• Publication Input Guide
• JuFo Browser

An Experimental Comparison of Cross-Validation Techniques for Estimating the Area Under the ROC Curve

Antti Airola, Tapio Pahikkala, Willem Waegeman, Bernard De Baets, Tapio Salakoski, An Experimental Comparison of Cross-Validation Techniques for Estimating the Area Under the ROC Curve. Computational Statistics and Data Analysis 55(4), 1828–1844, 2011.

http://dx.doi.org/10.1016/j.csda.2010.11.018

Abstract:

Reliable estimation of the classification performance of inferred predictive models is difficult
when working with small data sets. Cross-validation is in this case a typical strategy for
estimating the performance. However, many standard approaches to cross-validation suffer
from extensive bias or variance when the area under the ROC curve (AUC) is used as the
performance measure. This issue is explored through an extensive simulation study. Leave-
pair-out cross-validation is proposed for conditional AUC-estimation, as it is almost
unbiased, and its deviation variance is as low as that of the best alternative approaches.
When using regularized least-squares based learners, efficient algorithms exist for
calculating the leave-pair-out cross-validation estimate.

BibTeX entry:

@ARTICLE{jAiPaWaDeSa11a,
  title = {An Experimental Comparison of Cross-Validation Techniques for Estimating the Area Under the ROC Curve},
  author = {Airola, Antti and Pahikkala, Tapio and Waegeman, Willem and De Baets, Bernard and Salakoski, Tapio},
  journal = {Computational Statistics and Data Analysis},
  volume = {55},
  number = {4},
  pages = {1828–1844},
  year = {2011},
  keywords = {Area under the ROC curve; Classifier performance estimation; Conditional AUC estimation; Cross-validation; Leave-pair-out cross-validation},
}

Belongs to TUCS Research Unit(s): Algorithmics and Computational Intelligence Group (ACI), Turku BioNLP Group

Publication Forum rating of this publication: level 2

Edit publication