Images, language and games
Machine learning for computer vision
Machine learning for computer vision trains models to label images, locate objects with bounding boxes and outline regions pixel by pixel. It is used to inspect manufactured parts, count and track objects, and restore or measure images from cameras and scanners.
How machine learning is used in computer vision
Visual inspection and defect detection
Models learn what a good part looks like from camera images and flag scratches, dents or missing pieces on a production line. Because defects are rare, many systems train only on defect-free images and score how far a new image departs from them.
Real exampleMVTec ADMVTec published an industrial inspection benchmark of over 5,000 high-resolution images in fifteen object and texture categories, with defect-free training images and pixel-precise defect annotations.
Telling crops from weeds in the field
Cameras on farm machinery feed detection models that separate weeds from crop plants as the machine moves. The result drives an action, such as spraying only where a weed is found.
Real exampleJohn Deere See & SprayJohn Deere describes See & Spray as a factory-installed targeted sprayer that differentiates weeds from in-season crops and sprays just the weeds.
Segmenting objects pixel by pixel
Segmentation models outline each object exactly, which is what you need to measure areas, cut out products or count overlapping items. Large pretrained models now produce a mask from a single click or box.
Real exampleSegment Anything (Meta AI)Meta AI released the Segment Anything Model, a promptable segmentation model, together with a dataset of over one billion masks on 11 million images.
Reading medical images in research
Convolutional networks are trained on labelled scans or photographs to flag signs of disease, as a research aid to clinicians rather than a replacement. Results are validated against expert graders before any clinical use.
Real exampleGoogle diabetic retinopathy studyGoogle researchers presented a deep learning algorithm, published in JAMA, that interprets signs of diabetic retinopathy in retinal photographs.
Can you learn machine learning for computer vision?
Yes. If you can write basic Python and are comfortable with arrays, you can train a useful vision model within weeks by fine-tuning a pretrained network. Learn Python and NumPy first, then core machine learning and evaluation, then convolutional networks and transfer learning.
- Python, NumPy and images as arraysLoad images, resize and crop them, and treat each one as a height by width by channel array of numbers.Learn it in Python
- Linear algebra for visionVectors, matrices and convolution as a weighted sum explain what every layer of a vision model computes.Learn it in Maths
- Machine learning basics and evaluationTrain and test splits, overfitting, accuracy, precision and recall, and confusion matrices apply directly to image tasks.Learn it in Machine Learning
- Convolutional networks and transfer learningTrain small CNNs, then fine-tune pretrained backbones with data augmentation, which is how most real vision projects start.Learn it in Deep Learning
- Detection and segmentation in practiceUse a library such as Ultralytics or torchvision to train a detector on your own labelled boxes, and judge it with mean average precision.
Moving into machine learning for computer vision
If you already work in the field
If you already work with cameras, microscopes or inspection systems, you know the imaging conditions and what a real defect or object looks like, which is the hardest part to learn. Add Python and fine-tuning of pretrained models, then start by labelling a few hundred of your own images and training a classifier or detector on them.
If you come from data science or ML
If you already train tabular or text models, the new pieces are image data handling, augmentation, labelling tools and vision metrics such as IoU and mAP. Learn how lighting, lenses and camera placement change the data, and spend time with the people who capture and use the images.
Roles that do this workComputer vision engineerMachine learning engineer, perceptionImaging scientistApplied scientist, computer vision
Tools you will use
| Tool | What you use it for |
|---|---|
| OpenCV | Read, resize and transform images and video frames, and run classical vision steps around a model. |
| torchvision | Load pretrained classification, detection and segmentation models and standard image transforms in PyTorch. |
| Ultralytics | Train and run YOLO detection and segmentation models with a short Python or command-line workflow. |
| timm | Choose from many pretrained image backbones to fine-tune for classification or use as feature extractors. |
| Albumentations | Apply image augmentations that keep bounding boxes and masks aligned with the transformed image. |
Public data to practise on
- Oxford-IIIT Pet: Cat and dog photos labelled with breed, head boxes and pixel trimaps, so one dataset covers labels, boxes and masks.
- MVTec AD: Images of industrial objects and textures with and without defects, for learning to detect and outline faults.
- COCO: Everyday scenes annotated with object boxes, segmentation masks and captions, the standard detection benchmark.
Your first project
Download the Oxford-IIIT Pet dataset, fine-tune a pretrained ResNet-18 to classify the 37 breeds, and compare it with logistic regression on colour histograms using test accuracy.
Common questions
Is computer vision part of machine learning?
Computer vision is a field with its own classical methods, such as filtering and geometry, but most modern vision systems are built with machine learning. Deep learning models now do most recognition, detection and segmentation work.
How many images do I need to train a vision model?
With a pretrained backbone, a few hundred labelled images per class is often enough for a useful first model. Rare defects, unusual viewpoints and strict accuracy targets need more data and careful testing.
Should I learn OpenCV or deep learning first?
Learn enough OpenCV to load, resize, crop and draw on images, then move to deep learning. Classical OpenCV techniques stay useful for preprocessing and simple measurement tasks.
Do I need a GPU for computer vision?
You can learn and run small models on a CPU. Training or fine-tuning on thousands of images is much faster on a GPU, and free notebook GPUs are enough for learning projects.
Courses for these skills
- Deep Learning with PyTorchLearn image representations and reconstruction objectives.Syllabus available · lessons open as produced
- Practical Machine Learning in PythonBuild the foundations for classification, anomaly detection and model evaluation.51 published lessons · more lessons in production
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnKeep annotation coordinates and image identifiers aligned.23 published lessons · all planned lessons open
- Maths for Machine LearningConnect losses and gradients to pixel-level changes.26 published lessons · all planned lessons open
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.