Overfitting vs underfitting: examples and how to fix each
Tell overfitting from underfitting using training and validation error. Work through a demand example, diagnose the cause and choose a sensible next experiment.
Overfitting means a model fits the training examples well but performs poorly on unseen examples. Underfitting means it fails to capture useful patterns even in the training data. Compare training and validation results before deciding whether to simplify the model, improve its inputs or change how it learns.
The distinction matters because the same action can help one problem and worsen the other. Giving a model more flexibility may address underfitting; it can also make overfitting worse. Google's overfitting lesson explains the basic distinction and why performance on new examples is the goal.
Overfitting vs underfitting at a glance
| Pattern | Training error | Validation error | First questions to ask |
|---|---|---|---|
| Possible underfitting | High | Also high | Are useful inputs missing? Has training worked? Is the model too constrained? |
| Possible overfitting | Low | Substantially higher | Is the model learning details that do not transfer? Is the split valid? |
| Promising generalization | Useful performance | Similarly useful performance | Does the validation set represent the intended use? |
| Suspiciously good results | Very low | Very low | Have duplicates, future information or target-derived inputs leaked across the split? |
“High” and “low” need a reference: your baseline, metric and task requirements. There is no universal gap that proves overfitting. Small evaluation samples can also make results unstable.
A worked example: forecasting weekly orders
Imagine a shop predicting next week's orders. The following cases are teaching scenarios, not measured results.
An underfitting model: it predicts the same demand for every product and week, despite useful product and seasonal information being available. It misses patterns both in the historical training period and in later validation weeks. The next experiment might add relevant inputs or replace that constant baseline with a model able to use them.
An overfitting model: it follows nearly every fluctuation in the training period, including unusual one-off promotions. Its predictions fail in later weeks. The next experiment might limit model complexity and ask whether those promotion effects are supported by enough independent examples.
An invalid experiment: the input includes the final sales total from the week being predicted. Results look excellent because the answer is already in the data. Regularization cannot repair that experiment; remove the leaking feature and repeat the evaluation. See data leakage examples.
Use Series and DataFrames to inspect the underlying table. Use correlation, confounding and causation to question what an apparent association actually tells you.
How to read training and validation curves
Track both losses as training progresses. If training loss continues falling while validation loss starts rising, the model may be fitting details that do not help on unseen examples. Google's guide to interpreting loss curves also shows that noisy or stalled curves can point to training problems.
Keep the comparison meaningful. Training and validation must use compatible measurements. Some training procedures include penalties or random transformations that are absent during evaluation, so their reported losses need interpretation.
A curve plotted against training steps answers a different question from a learning curve plotted against the amount of training data. scikit-learn's learning-curve guide explains how training-set size can help diagnose whether additional examples may improve generalization.
For your project, record the axis, metric and data split beside every chart. A smooth line without that context cannot justify changing the model.
How to address overfitting
Try one change at a time and compare it with the unchanged baseline:
- Reduce flexibility. For a tree, try shallower depth or larger leaves. For a neural network, consider a smaller model if the current one is unnecessarily large.
- Use suitable regularization. This adds constraints to training. Google's L2 regularization lesson explains the tradeoff between fitting the data and limiting complexity.
- Use validation evidence to stop training. For iterative training, select a checkpoint based on validation behaviour rather than always taking the final one.
- Improve data coverage. Seek examples of the cases the model fails on. Repeating copies of existing rows does not add independent evidence.
Choose among these using an error review. If failures concentrate on a new product category, investigate coverage. If failure appears only after a preprocessing change, audit that change before reducing the network size.
How to address underfitting
First check that the experiment is functioning. Are labels aligned with inputs? Is the training objective appropriate? Does the model receive the columns you think it receives? Can it learn anything useful on a small development subset?
Then try improvements that match the diagnosis: supply a missing informative feature, relax excessive regularization, allow sufficient optimization or use a model able to capture the relevant relationship. More training will not rescue an input table with no useful signal.
Learn what a parameter update is doing in gradient descent. If the derivative is unfamiliar, study Derivatives: Rate of Change and the Tangent first.
Keep the final test separate from debugging
Use training data to fit parameters and validation data to choose your approach. Once you have fixed the choices, evaluate on a separate test set. Repeatedly changing the model in response to test performance makes that set part of development.
For the small project in this article, keep an experiment log with the inputs, split, model settings, training result, validation result and reason for each change. Record failed attempts as well as improvements. That log makes a portfolio project easier to assess than a single best score.
Common questions
Does high training accuracy prove overfitting?
No. Compare it with performance on valid held-out examples and with the baseline. High training accuracy alone says little about generalization.
Does more data always fix overfitting?
No. Additional relevant, independent examples may help. Copies, incorrectly labelled examples or examples unrelated to the intended use may not.
Can deep learning models underfit?
Yes. Poor inputs, unsuitable settings or unsuccessful optimization can prevent a neural network from learning useful patterns. Deep learning does not remove the need to diagnose training.
Practise the complete workflow
Start with estimators: fit, predict and transform, then pipelines. Continue with the Statistics course to understand sampling and uncertainty, and the Machine Learning syllabus for model development.
Primary references were checked on 12 October 2026.
