ML 4.5Lesson outlineLesson 5 of 9 in Module 4
Hyperparameter Search
Choose hyperparameters with a search that spends a fixed budget well, and report a score the search did not bias.
Video in production.
Lesson outline: Hyperparameter Search
In this lesson9 parts
- 01Three hyperparameters
- 02The training tumors
- 03Try every setting
- 04Random settings, same budget
- 05Halving and a model of the search
- 06The winner's score is optimistic
- 07Where searches mislead
- 08Code
- 09Recap, your challenge, and what comes next
Key terms: Hyperparameter Search
The words this lesson introduces, each in one line. The module’s glossary collects them all, and the AI glossary has every term across the courses.
- grid search
- Trying every combination of a set of hyperparameter values.
- random search
- Trying randomly sampled settings.
- search budget
- The number of fits a search may use.
- successive halving
- Running many settings on little data and giving more data to the best.
- Bayesian optimization
- Choosing each next setting from the scores of the settings tried so far.
- nested cross-validation
- An outer cross-validation that scores a whole search run inside each outer fold.
- one-standard-error rule
- Choose the simplest setting within one standard error of the best.
Quiz: Hyperparameter Search
Your first pick on each question is the one that counts. Getting one wrong here is how the lesson sticks.