• TUCS Publication Series
• How to Publish in TUCS Series
• Publication Search
• Publication Graph
• Publication Input
• Publication Input Guide
• JuFo Browser

Critical Points in Assessing Learning Performance Via Cross-Validation

Hanna Suominen, Tapio Pahikkala, Tapio Salakoski, Critical Points in Assessing Learning Performance Via Cross-Validation. In: Matti Pöllä Mari-Sanna Paukkeri Olli Simula Timo Honkel (Ed.), Proceedings of AKRR\'08, the 2nd International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reasoning, 9-22, Multiprint, Espoo, Finland, 2008.

Abstract:

Quality assessment of learning methods is essential
when adapting them to different tasks,
introducing new algorithms,
developing the existing ones,
or tracking the learning improvements over time.
Obtaining realistic results is, however, complicated.
In this paper, we clarify this
by addressing performance evaluation measure and method selection,
with a main focus on combining
the AUC measure with the cross-validation method.
We conclude that it is crucial to choose both the measure and method
that reflect the evaluation aspects, learning task and data in question. Furthermore, the evaluation setting must correspond to
the intended use environment and the test set has to be completely
independent from the creation of the learner.
Finally, special caution is needed when forming the cross-validation folds.

Files:

Full publication in PDF-format

BibTeX entry:

@INPROCEEDINGS{inpSuPaSa08a,
  title = {Critical Points in Assessing Learning Performance Via Cross-Validation},
  booktitle = {Proceedings of AKRR\'08, the 2nd International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reasoning},
  author = {Suominen, Hanna and Pahikkala, Tapio and Salakoski, Tapio},
  editor = {Timo Honkel, Matti Pöllä Mari-Sanna Paukkeri Olli Simula},
  publisher = {Multiprint, Espoo, Finland},
  pages = {9-22},
  year = {2008},
}

Belongs to TUCS Research Unit(s): Turku BioNLP Group

Edit publication