Evaluating machine learning models and their diagnostic value - ARAMISLAB - Algorithms, models and methods for images and signals of the human brain Access content directly
Book Sections Year : 2023

Evaluating machine learning models and their diagnostic value

Abstract

This chapter describes model validation, a crucial part of machine learning whether it is to select the best model or to assess risk of a given model. We start by detailing the main performance metrics for different tasks (classification, regression), and how they may be interpreted, including in the face of class imbalance, varying prevalence, or asymmetric cost-benefit trade-offs. We then explain how to estimate these metrics in a unbiased manner using training, validation, and test sets. We describe cross-validation procedures –to use a larger part of the data for both training and testing– and the dangers of data leakage –optimism bias due to training data contaminating the test set. Finally, we discuss how to obtain confidence intervals of performance metrics, distinguishing two situations: internal validation or evaluation of learning algorithms, and external validation or evaluation of resulting prediction models.
Fichier principal
Vignette du fichier
Chapter 20 - Final.pdf (783.17 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
licence : CC BY - Attribution

Dates and versions

hal-03682454 , version 1 (31-05-2022)
hal-03682454 , version 2 (01-06-2022)
hal-03682454 , version 3 (02-06-2022)
hal-03682454 , version 4 (21-01-2023)
hal-03682454 , version 5 (20-04-2023)

Identifiers

  • HAL Id : hal-03682454 , version 5

Cite

Gaël Varoquaux, Olivier Colliot. Evaluating machine learning models and their diagnostic value. Olivier Colliot. Machine Learning for Brain Disorders, Springer, 2023. ⟨hal-03682454v5⟩
11672 View
3162 Download

Share

Gmail Facebook X LinkedIn More