SchoolWhool
Quizzes and saved progress, freeSign in freeSign in
Machine Learning · Module 2

Exploring a Dataset for Modelling

Take each lesson, answer its quiz, then do its practice. After the last lesson come the module quiz, the project and the interview questions.

Lessons

  1. 2.1Features Against the TargetUse per-class distributions and pairwise views to decide which features help predict the target, and which repeat each other.
  2. 2.2The Modelling Checks: Target, Leakage and Feature DecisionsBefore modelling an unseen table, check the target, find the columns that must not be used, and write the decisions a modeller acts on.

Module material

Take these after the last lesson, in this order. Next on each page leads through them.

  1. Module quiz12 questions across the whole module. Take it after the last lesson; your first pick on each question is the one that counts.
  2. Project: A Data Report for UCI Online Shoppers Purchasing IntentionStated as a problem, with no template and no step-by-step instructions. Working out the steps is the point.
  3. NotebooksOpen them in Google Colab. Each answer is checked as you go: correct, wrong with the expected value, or not answered yet.
  4. Cheat sheetOne page of the module’s terms, rules and gotchas, with the output each rule produces.
  5. Interview questions12 questions interviewers ask on this module’s topics, from first principles to the follow-ups. Each comes with a model answer.

Glossary

Every term the module introduces, with the lesson that introduces it.

histogram, density (reminder, Stats 1.1)
Counts of values in bins; with density on the vertical axis, each class's bars have the same total area, so a larger class does not look more important. 2.1
class-conditional distribution
The distribution of a feature within one class: concave_points_worst among the malignant tumors. 2.1
overlap
The region where two class distributions share values. 2.1
signal
Information a feature carries about the label; the less the class distributions overlap, the more signal. 2.1
empirical CDF (reminder, Stats 1.2)
For each value, the fraction of a group at or below it. 2.1
CDF gap (narration)
The largest vertical distance between the two classes' empirical CDFs: 0.812 for concave_points_worst. A threshold placed there separates the classes as well as the feature can. 2.1
Pearson correlation (reminder, Stats 1.6 and 1.9)
A number from −1 to 1 measuring how closely two features follow a straight line together. 2.1
pair plot, heatmap (reminder, Stats 1.10)
A scatter for every pair of features; a grid of cells coloured by each pair's correlation. 2.1
redundant feature
A feature whose information another feature already carries: perimeter_mean beside radius_mean. 2.1
summary statistic
One number computed from a whole data set, such as a correlation; it can not show the shape behind it. 2.1
skewed (narration)
A distribution with a long tail on one side, whose few large values pull its mean away from its median: area_se, median 24.61, largest 542.2. 2.1
prediction time
The moment the model is used: for the bank, before anyone dials. 2.2
target leakage
A feature carrying label information that is not available at prediction time: duration. 2.2
data leakage
Information in training that the model will not have when it is used; target leakage is one kind. 2.2
sentinel value
A special value that stands for a missing or unusual case, not an amount: pdays = 999. 2.2
duplicate (reminder, ML 1.5)
A row whose content repeats another row; the bank file has 12. 2.2
data report
The findings about a table, each with the decision it leads to: keep, transform, drop, or a question for a later video. 2.2