Transfer learning vs fine-tuning: an image project guide
Understand transfer learning, feature extraction and fine-tuning. Plan an image classification project with frozen weights, valid splits and a fair comparison.
Transfer learning reuses knowledge from a previously trained model for a new task. Fine-tuning is one way to do it: continue training some or all of the pretrained parameters on new data. Another approach keeps the pretrained model frozen and trains a new classifier on its features.
For a first image project, our recommendation is to establish a frozen-feature baseline before fine-tuning. You can then ask whether updating pretrained weights improves a valid held-out evaluation enough to justify the additional work.
Transfer learning, feature extraction and fine-tuning
| Approach | What changes during training? | What are you investigating? |
|---|---|---|
| Training from scratch | A newly initialized model's parameters | Can this dataset support learning the whole representation? |
| Frozen feature extraction | The new classifier; the pretrained feature extractor stays fixed | Are the existing representations useful for the new categories? |
| Fine-tuning | Some or all pretrained parameters, usually alongside the classifier | Does adapting the representations improve the new task? |
“Transfer learning vs fine-tuning” can sound like a choice between unrelated methods. In this setting, fine-tuning is a form of transfer learning. Keras's transfer learning guide explains freezing layers, adding a new output head and later unfreezing weights.
The distinction is about parameter updates. Loading a pretrained model without changing its parameters is different from adapting those parameters through training.
A concrete project: classify objects in photographs
Suppose you want to sort photographs of household objects into categories. This is a suggested learning project, not a completed SchoolWhool experiment. Choose images you are allowed to use, define the categories and record how each label was assigned.
Write down the intended use: will the classifier see photographs of entirely new objects, different photographs of familiar objects, or both? That decision controls the evaluation split.
For the new-object case, group photographs by physical object before partitioning them. Several photos of one mug should not be spread across training and evaluation. Otherwise a result may partly measure recognition of objects the model has already seen. Our data leakage guide explains that boundary.
Create a manifest containing each file, its category, its object group and its split. Keep the final test fixed while you develop the model. In your report, include categories that are easily confused and the lighting or backgrounds that cause errors.
Experiment 1: keep pretrained weights frozen
Choose a pretrained image model with documented input requirements. Replace or attach its task-specific classifier, then train that classifier while keeping the feature extractor fixed. PyTorch's computer vision transfer learning tutorial demonstrates this approach alongside fine-tuning.
“Frozen” means the parameters are not being updated. Framework details such as evaluation mode and normalization state still matter; follow the official recipe for the model you use.
For your experiment log, save the model identity, weight version, input preprocessing, split manifest and classifier settings. This makes a later comparison possible. Without those details, “pretrained model” is too vague to reproduce.
Learn Shape, dtype and Reshaping before changing image dimensions. Then study vectorised operations and broadcasting so channel normalization and batches are easier to reason about.
Experiment 2: fine-tune selected weights
After establishing the first experiment, unfreeze the layers your recipe intends to adapt and continue training. Use validation evidence to decide whether the extra changes help. Keep the starting checkpoint, splits and comparison metric clear.
TensorFlow's transfer learning tutorial describes using a lower learning rate when fine-tuning and handling batch-normalization layers carefully. These details matter because you are adjusting an existing representation rather than beginning with new parameters.
If you use Keras, changing which layers are trainable requires recompiling the model for those changes to affect subsequent fit calls. Follow the Keras fine-tuning instructions instead of assuming that changing a flag is the entire procedure.
Avoid treating fine-tuning as automatic improvement. It can overfit a small or narrow dataset. Compare the frozen and adapted models on the same validation examples before choosing one for the final test.
To understand what those updates do, study gradient descent and derivatives. To diagnose a worsening validation result, use overfitting vs underfitting.
How to compare the two experiments fairly
For the household-object project, keep a record like this. Fill it with your measured observations rather than copying an expected result from a tutorial.
| Item to record | Frozen-feature experiment | Fine-tuning experiment |
|---|---|---|
| Dataset and split manifest | Exact version used | Same version for comparison |
| Starting pretrained weights | Model and checkpoint identity | Document whether you start from the trained first experiment |
| Updated parameters | Classifier parameters | Classifier plus named unfrozen layers |
| Selection evidence | Validation metric and error review | Same metric and review process |
| Training resources | Measured time and hardware | Measured time and hardware |
| Final test | Evaluate the selected approach after development | Keep it out of development decisions |
Include a simple baseline, such as always choosing the most common class, and inspect errors by category. If a category is rare, an overall accuracy alone may hide poor performance on it. Study cross-entropy and log loss to distinguish the training objective from the decisions you care about.
If fine-tuning barely changes the relevant outcome but increases costs, explain why the frozen approach may be preferable for your task. If it helps, show the evidence and where errors remain. Either conclusion can make a useful portfolio report.
Common questions
Is fine-tuning always better than frozen feature extraction?
No. The answer depends on the starting representation, dataset, training procedure and evaluation. Establish the frozen baseline and measure the change.
How many labelled images do I need?
There is no universal count. Coverage, category ambiguity, label consistency and similarity to the pretrained model's original task matter. Start with an evaluation plan and inspect what additional examples are needed.
Does changing a prompt count as fine-tuning?
Changing a prompt changes the input, not the model's parameters. Fine-tuning uses training to update parameters. The activities can solve different needs and should be described separately.
Should I use PyTorch or TensorFlow for transfer learning?
Both have official image-transfer tutorials. Choose the maintained example closest to your task and environment. Our PyTorch vs TensorFlow guide explains the beginner workflow choices.
Build the foundations, then document your project
Use the Python course to organize the experiment, the Data Tools course to prepare its inputs, and the Machine Learning course to understand optimization. Read the portfolio project briefs for a broader view of reproducible deliverables.
SchoolWhool's five published curriculum tracks build those foundations. Its deep learning tutorial guide describes further study; it does not imply that a separate Deep Learning track has been released.
Primary references were checked on 12 October 2026.
