Images, language and games
Deep learning for unsupervised
Deep learning for unsupervised learning trains networks without labels, using autoencoders, contrastive learning and other self-supervised objectives. The learned representations support clustering, anomaly detection, similarity search and strong starting points for tasks with few labels.
How deep learning is used for unsupervised learning
Learning image features without labels
Contrastive methods train a network to give similar vectors to two views of the same image and different vectors to other images. The learned features then need few labels for a new task.
Real exampleSimCLR (Google)SimCLR is a simple framework for contrastive learning of visual representations without labels.
General-purpose visual features
Large self-supervised models produce image embeddings that work for retrieval, clustering and classification without fine-tuning. Teams use them as a shared backbone across many tasks.
Real exampleDINOv2 (Meta AI)Meta AI showed that self-supervised pretraining on enough curated data produces all-purpose visual features that work across tasks without finetuning.
Anomaly detection from normal examples
When defects are rare, a model is fitted only on normal images and flags anything that looks different. Deep features make this work far better than raw pixels.
Real examplePatchCorePatchCore detects and localises industrial defects using a memory bank of patch features from defect-free images only, evaluated on the MVTec AD benchmark.
Masked reconstruction pretraining
Hiding parts of the input and training a network to fill them in is a simple, scalable unsupervised objective. Autoencoders trained this way give strong starting points for vision tasks.
Real exampleMasked Autoencoders (Meta AI)Masked autoencoders mask random patches of an image and reconstruct the missing pixels to pretrain vision transformers.
Pretraining language models
Language models learn from raw text by predicting hidden or next words, with no human labels. This self-supervised step underlies modern NLP.
Real exampleBERT (Google)BERT is pretrained on unlabelled text with a masked language modelling objective and then fine-tuned for downstream tasks.
Can you learn unsupervised deep learning?
Yes, once you are comfortable with supervised deep learning. Learn Python, linear algebra and classical unsupervised methods such as PCA and k-means, then autoencoders, then contrastive and masked self-supervised learning.
- Linear algebra and PCAVectors, distances and projections explain embeddings and give you the classical baseline to beat.Learn it in Maths
- Classical unsupervised learningClustering, dimensionality reduction and anomaly detection with scikit-learn show what a good representation should do.Learn it in Machine Learning
- Neural networks and autoencodersTrain an autoencoder on unlabelled images and inspect its codes and reconstructions.Learn it in Deep Learning
- Self-supervised methodsTrain or load contrastive and masked models, and evaluate them with a linear probe on a small labelled set.
Moving into self-supervised and unsupervised learning
If you already work in the field
If you have large amounts of unlabelled data, such as images, sensor readings or documents, you already have the raw material. Start with pretrained self-supervised embeddings, then cluster or search them and check results with domain experts.
If you come from data science or ML
If you know supervised deep learning, learn the self-supervised objectives, data augmentations and evaluation protocols such as linear probing and k-nearest-neighbour accuracy. The hard part is judging representation quality without labels.
Roles that do this workMachine learning research scientistMachine learning engineerComputer vision engineerApplied scientist, representation learning
Tools you will use
| Tool | What you use it for |
|---|---|
| PyTorch | Build and train autoencoders and self-supervised models with full control over the objective. |
| LightlySSL | Ready PyTorch implementations of self-supervised methods such as SimCLR and MoCo. |
| scikit-learn | PCA, k-means and linear probes to compare against and evaluate learned representations. |
| UMAP | Project embeddings to two dimensions to inspect clusters alongside quantitative checks. |
| DINOv2 in Transformers | Use pretrained self-supervised image features without training your own encoder. |
Public data to practise on
- Fashion-MNIST: Greyscale clothing images in ten classes, small enough to train autoencoders and linear probes quickly.
- CIFAR-10: Small colour images whose labels you hold back during pretraining, then use to evaluate features with a linear probe.
Your first project
Download Fashion-MNIST, train a small autoencoder on the training images without labels, fit logistic regression on its frozen codes, and compare with PCA features plus logistic regression using test accuracy.
Common questions
What is the difference between unsupervised and self-supervised learning?
Unsupervised learning uses no labels at all. Self-supervised learning is a kind of unsupervised learning that creates its own training targets from the data, such as hidden patches or words.
How do you evaluate a model trained without labels?
Usually with a small labelled set: train a linear classifier or k-nearest-neighbour on the frozen features and measure accuracy. Clustering quality and retrieval results are also checked.
Are autoencoders still useful?
Yes. They remain simple, effective tools for compression, denoising and anomaly detection, and masked autoencoders are a strong modern pretraining method.
Courses for these skills
- Deep Learning with PyTorchStudy representation-learning objectives and neural training.Syllabus available · lessons open as produced
- Maths for Machine LearningWork with norms, dot products and vector geometry.26 published lessons · all planned lessons open
- Practical Machine Learning in PythonStudy distances, clustering and evaluating unsupervised results.51 published lessons · more lessons in production
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnInspect feature scales and representative rows.23 published lessons · all planned lessons open
- Generative AI: LLMs, RAG and AgentsConnect embeddings and retrieval to language-model applications.Syllabus available · lessons open as produced
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.