ML 4.4Lesson outlineLesson 4 of 9 in Module 4
Cross-Validation
Estimate how a model will do on unseen data from the training set alone, with a split that matches how the data was collected.
Video in production.
Lesson outline: Cross-Validation
In this lesson9 parts
- 01One split, twenty seeds
- 02The training tumors again
- 03Keep the luckiest seed
- 04Every tumor validated once
- 05Stratified folds, and the spread
- 06Reading a cross-validated score
- 07Folds that leak
- 08Code
- 09Recap, your challenge, and what comes next
Builds on
This lesson stands on its own. These go deeper into what it uses.
Key terms: Cross-Validation
The words this lesson introduces, each in one line. The module’s glossary collects them all, and the AI glossary has every term across the courses.
- cross-validation
- Estimating a model's score by fitting and validating on several splits of the training rows.
- k-fold
- k splits, each part validating once.
- fold
- One of the k parts.
- stratified k-fold
- Folds that keep each class's share.
- standard error (reminder, Stats 5.1)
- How much an estimate moves from one sample to the next.
- standard error of a cross-validated score
- The fold scores' standard deviation over the square root of the number of folds.
- group k-fold
- Folds that keep each group's rows together.
- time-based split
- Fitting on earlier rows and validating on later ones.
Quiz: Cross-Validation
Your first pick on each question is the one that counts. Getting one wrong here is how the lesson sticks.