Data and modeling
Machine learning workflow
A machine learning workflow is the repeatable path from raw data to a model you can trust: load and split the data, build features, fit a baseline and candidates, tune with cross-validation and report one final test score. Pipelines keep every step reproducible.
How a machine learning workflow is used in real projects
Preventing data leakage with pipelines
Scaling, imputation and encoding must be learned from training data only and then applied to test data. Putting every step in a pipeline makes cross-validation repeat them correctly in each fold.
Real examplescikit-learn: Common pitfalls and recommended practicesThe scikit-learn documentation shows how inconsistent preprocessing and data leakage occur and recommends Pipeline to chain transformations with estimators.
Starting with a simple model
A simple first model gets the data, features and evaluation working end to end, and gives a score every later model must beat. Most early problems are in the pipeline, not the algorithm.
Real exampleRules of Machine Learning (Google)Rule 4 of Google's guide is to keep the first model simple and get the infrastructure right.
Choosing and tuning candidates fairly
Comparing many models and settings on the same validation data can overfit that data. Nested cross-validation or a final untouched test set keeps the reported score honest.
Real exampleOn Over-fitting in Model Selection (Cawley and Talbot, JMLR 2010)The paper shows that over-fitting the model selection criterion can bias performance estimates and affect which model is chosen.
Picking model families for tabular data
For tables of mixed features, tree ensembles are usually the first strong candidates after a linear baseline. Benchmarks help decide where neural networks are worth the extra effort.
Real exampleWhy do tree-based models still outperform deep learning on tabular data? (Grinsztajn et al., 2022)On a benchmark of 45 datasets, the authors find that tree-based models remain state of the art on medium-sized tabular data of around 10,000 samples.
Avoiding leakage in research
The same workflow mistakes, such as preprocessing before splitting or features that encode the target, appear in published research. A careful workflow is what makes a reported result reproducible.
Real exampleLeakage and the Reproducibility Crisis in ML-based Science (Kapoor and Narayanan, 2023)The authors survey fields that adopted machine learning and show that data leakage is a widespread problem that has led to reproducibility failures.
Can you learn the machine learning workflow?
Yes, and it is the best place to start, because the same workflow applies to almost every supervised problem. Learn Python and pandas, then the scikit-learn fit and predict pattern, then pipelines, cross-validation and tuning.
- PythonLearn the language basics, functions and data structures used throughout ML code.Learn it in Python
- Data tools and the scikit-learn APILoad and explore data with pandas and learn the fit, predict and transform pattern shared by all scikit-learn objects.Learn it in Data Tools
- Measuring a modelSplit data, choose metrics, compare with a baseline and use cross-validation to estimate performance.Learn it in Machine Learning
- Pipelines and tuningCombine preprocessing and model in a Pipeline, tune it with grid search or Optuna, and report one final test score.Learn it in Machine Learning
Where to go after learning the workflow
If you already work in the field
Analysts and scientists who already run reproducible analyses have the right instincts. Rebuild one of your analyses as a scikit-learn Pipeline with cross-validation, and record the baseline and final test score.
If you come from data science or ML
If you know the algorithms but work ad hoc in notebooks, adopt the workflow as code: one script or pipeline from raw data to a logged score. Then add experiment tracking so results can be compared and repeated.
Roles that do this workData scientistMachine learning engineerApplied scientistResearch engineer
Tools you will use
| Tool | What you use it for |
|---|---|
| scikit-learn | Chain preprocessing and models in Pipelines and evaluate them with cross-validation. |
| pandas | Load raw files, fix types and split off the target before modelling. |
| Optuna | Tune hyperparameters within a set budget and keep each study reproducible. |
| MLflow | Log parameters, metrics and artefacts so each run can be reproduced. |
| Jupyter | Explore data interactively before moving stable steps into scripts. |
Public data to practise on
- Adult (UCI): Census records labelled by whether income exceeds 50K a year, a standard benchmark for a full classification workflow.
Your first project
Download the Adult dataset, predict whether income exceeds 50K with a Pipeline of one-hot encoding and logistic regression, and compare it with a majority-class baseline using balanced accuracy under 5-fold cross-validation.
Common questions
What are the steps in a machine learning workflow?
Define the problem and target, collect and explore data, split it, build features, train a baseline and candidates, tune with cross-validation, evaluate once on a test set, then deploy and monitor.
What is the difference between a validation set and a test set?
The validation set, or cross-validation, is used to choose models and settings. The test set is used once at the end to estimate performance on new data, and should not influence any choices.
Why use a scikit-learn Pipeline?
A Pipeline bundles preprocessing and model so they are fitted together on training data only. This prevents leakage during cross-validation and makes the whole model easy to save and reuse.
Courses for these skills
- Python for AI and Machine LearningWrite functions and inspect exceptions while building a repeatable analysis.33 published lessons · all planned lessons open
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnUnderstand table columns, array shapes and row alignment.23 published lessons · all planned lessons open
- Practical Machine Learning in PythonDesign model comparisons that isolate the intended change.51 published lessons · more lessons in production
- Statistics and Probability for Machine LearningAssess sampling variability and uncertainty.41 published lessons · all planned lessons open
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.