Health and life sciences
Machine learning for X ray
Machine learning on X-ray images classifies findings such as pneumonia or effusion, localises the region behind a finding, and flags poor-quality images. Research models are trained with convolutional networks or vision transformers on labelled chest X-ray collections.
How machine learning is used on X-ray images
Classifying findings in chest radiographs
A convolutional network takes a radiograph and outputs a probability for each of several findings. Research results are reported as areas under the ROC curve on a held-out set labelled by radiologists.
Real exampleCheXNet (Rajpurkar et al., 2017)A 121-layer convolutional network trained on the ChestX-ray14 dataset to detect pneumonia and 13 other findings, compared with four practising academic radiologists on an annotated test set.
Building labelled datasets at scale
Labels for large collections are extracted automatically from radiology reports, which is what makes training possible but also introduces uncertainty. Public datasets document how labels were derived so results can be compared.
Real exampleCheXpert (Irvin et al., 2019)A dataset of 224,316 chest radiographs from 65,240 patients, labelled for 14 observations by an automatic labeller that captures uncertainty in radiology reports.
Testing whether a model transfers between hospitals
A model that scores well where it was trained can fall off on images from another site, because scanners, patient mix and disease prevalence differ. External validation is now a standard expectation in this research.
Real exampleGeneralisation of chest radiograph models (Zech et al., 2018)Found that models trained at one hospital system performed worse on data from another, and that convolutional networks could identify the hospital system and department from the image itself.
Assessment by health bodies
Health technology bodies review commercial software and say what evidence is still needed before routine use. Reading these assessments shows which claims hold up outside a research paper.
Real exampleNICE HealthTech guidance on AI for fracture detectionNICE guidance states that four named AI technologies can be used in the NHS during an evidence generation period as options to help healthcare professionals detect fractures on X-rays in urgent care, subject to regulatory and NHS England criteria.
Screening programmes in global health
Computer-aided detection reads chest radiographs where radiologists are scarce, for example in tuberculosis screening. Its place in a pathway is set by guidance, not by benchmark scores.
Real exampleWHO consolidated guidelines on tuberculosis screeningThe WHO guidelines on systematic screening for tuberculosis disease list computer-aided detection software among the recommended screening tools alongside symptom screening, chest radiography and rapid diagnostic tests.
Can you learn machine learning for X-ray images?
Yes. This is a well-documented area with public datasets and standard architectures, so a motivated learner can build a credible research project. Learn Python and image handling, then classical evaluation, then convolutional networks and transfer learning, and finally the evaluation practices that separate a benchmark score from a clinically meaningful one.
- Python and image dataRead DICOM and PNG images, build datasets and loaders, and keep patient identifiers with every image.Learn it in Python
- Evaluation and statisticsLearn ROC and precision-recall curves, confidence intervals and calibration, and why prevalence changes what a score means.Learn it in Statistics
- Convolutional networks and transfer learningFine-tune a pretrained image model on a labelled radiograph set, with augmentation suited to radiographs.Learn it in Deep Learning
- Honest validationUse patient-disjoint splits, then test on a dataset from another source to see how much performance drops.Learn it in Machine Learning
- Context and governanceRead how imaging software is assessed and regulated, so you understand the distance between a research result and clinical use.
Moving into medical imaging machine learning
If you already work in the field
Radiologists and radiographers know how images are acquired, how reports are written and where labels come from. Add Python and deep learning, then work with a research team on a question from your own department, using data governance and ethics approval from the start.
If you come from data science or ML
Engineers from general computer vision should learn how radiographs are acquired and stored, how reports become labels and why site-specific artefacts fool models. Start on public datasets, always evaluate externally, and present results as research rather than diagnosis.
Roles that do this workMedical imaging research scientistMachine learning engineer, medical imagingClinical data scientistImaging research analyst
Tools you will use
| Tool | What you use it for |
|---|---|
| PyTorch | Train convolutional networks and vision transformers on image tensors. |
| torchvision | Load pretrained image backbones and apply augmentations for transfer learning. |
| MONAI | Use medical imaging transforms, networks and metrics built for research on scans and radiographs. |
| TorchXRayVision | Load common chest X-ray datasets with consistent labels and start from pretrained X-ray models. |
| pydicom | Read DICOM files and their metadata before converting images to arrays. |
Public data to practise on
- MedMNIST: Small, standardised medical image sets including PneumoniaMNIST and ChestMNIST, with fixed train and test splits.
- CheXpert: Chest radiographs labelled for common findings, including uncertain labels; download is through the Stanford AIMI site after free sign-up and accepting its research use terms.
- MIMIC-CXR: Chest radiographs with free-text reports; access requires PhysioNet credentialing, training and a data use agreement.
Your first project
Download PneumoniaMNIST from MedMNIST, train a small convolutional network to classify pneumonia versus normal, and compare it with logistic regression on raw pixels using ROC AUC on the official test split.
Common questions
Can a model trained on public X-ray data be used on patients?
No. Public benchmark results show research performance only. Clinical use requires regulatory approval, local validation and evidence that it helps care, which is why health bodies publish separate assessments.
Which public chest X-ray datasets are suitable for learning?
ChestX-ray14, CheXpert and MIMIC-CXR are the usual starting points. Read each one's label definitions first, because labels come from reports and carry uncertainty.
Why do radiograph models drop in accuracy at a new hospital?
Scanners, positioning, patient mix and disease prevalence differ, and models can latch onto site-specific markers. Published work shows models can even identify which hospital an image came from.
Do you need a lot of GPU time?
A single GPU is enough to fine-tune a pretrained model on a public dataset, and smaller image sizes make experiments faster while you learn.
Courses for these skills
- Deep Learning with PyTorchStudy CNNs and object detection in the Computer Vision syllabus.Syllabus available · lessons open as produced
- Statistics and Probability for Machine LearningDistinguish repeated observations from independent evidence.41 published lessons · all planned lessons open
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnKeep annotation coordinates and image identifiers aligned.23 published lessons · all planned lessons open
- Practical Machine Learning in PythonChoose an evaluation design before tuning a model.51 published lessons · more lessons in production
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.