Science and engineering
Machine learning for inverse problems and data assimilation
Machine learning speeds up inverse problems by learning to reconstruct images from incomplete measurements, as in CT and MRI, and by providing fast surrogate models and learned corrections inside data-assimilation systems that combine observations with forecasts.
How machine learning helps inverse problems and data assimilation
Reconstructing images from fewer measurements
Networks learn to reconstruct images from incomplete data, such as undersampled MRI or low-dose CT, by learning what realistic images look like. The aims are faster scans or lower doses, and reconstructions must be checked for invented detail.
Real examplefastMRI (Zbontar et al., 2018)Released a large dataset of raw MRI measurements and clinical images, with benchmarks for machine learning reconstruction from undersampled data, for research use.
Learned iterative reconstruction
Unrolled methods take a classical iterative algorithm, keep the known physics of the measurement and replace some steps with trained networks. This combines the reliability of the forward model with learned prior knowledge.
Real exampleLearned Primal-Dual (Adler and Öktem, 2017)Unrolled a primal-dual optimisation method for tomographic reconstruction, replacing its proximal operators with convolutional networks trained end to end on raw data.
Learning models inside data assimilation
Data assimilation blends observations with model forecasts to estimate the current state of a system, as in weather prediction. Machine learning can emulate or correct the model inside this cycle, learning from the assimilated states.
Real exampleCombining data assimilation and machine learning (Brajard et al., 2020)Alternated an ensemble Kalman filter with a neural network to learn an emulator of hidden, possibly chaotic, dynamics from sparse and noisy observations.
Inferring parameters with simulators
When a simulator can generate data but its likelihood cannot be written down, neural networks can learn the link between parameters and simulated data and then infer parameters from real observations.
Real exampleThe frontier of simulation-based inference (Cranmer, Brehmer and Louppe, 2020)Reviewed simulation-based inference methods for drawing conclusions from complex simulators and described how machine learning is expanding what they can do.
Can you learn machine learning for inverse problems?
Yes, but it rewards strong mathematics. Learn Python, then linear algebra, optimisation and probability, then regularisation and Bayesian estimation, then deep learning, and finally unrolled methods and learned priors.
- Python and NumPyImplement simple forward operators and solvers such as gradient descent on least squares.Learn it in Python
- Linear algebra and optimisationIll-posedness, regularisation and adjoints are the foundation of every inverse method.Learn it in Maths
- Probability and Bayesian inferenceData assimilation and uncertainty quantification are Bayesian estimation problems.Learn it in Statistics
- Deep learningTrain convolutional networks, then unrolled networks that include the forward operator.Learn it in Deep Learning
- Forward models of your applicationLearn the physics of your measurement, such as the Radon transform for CT or Fourier sampling in MRI.
Moving into machine learning for inverse problems
If you already work in the field
Imaging scientists, geophysicists and meteorologists who work with inverse problems already know regularisation, adjoints and error statistics. Add PyTorch and implement a learned reconstruction or learned correction on a benchmark, comparing it with your classical method. Medical imaging companies, weather centres and research labs hire for this mix.
If you come from data science or ML
Machine learning practitioners need the theory of ill-posed problems, regularisation and Bayesian estimation, plus the forward model of the application. An inverse problems or data assimilation textbook together with an open benchmark such as fastMRI gives the necessary grounding.
Roles that do this workImaging scientistData assimilation scientistResearch scientist, computational imagingScientific machine learning engineer
Tools you will use
| Tool | What you use it for |
|---|---|
| DeepInverse | Train and evaluate learned reconstruction methods for imaging inverse problems. |
| DAPPER | Benchmark Kalman, ensemble and variational data-assimilation methods on test models such as Lorenz-96. |
| CUQIpy | Quantify uncertainty in inverse problems with Bayesian modelling and sampling. |
| SciPy | Solve regularised least-squares and optimisation problems as classical baselines. |
| PyTorch | Build learned reconstruction networks and differentiable forward operators. |
Public data to practise on
- LoDoPaB-CT: Simulated low-dose CT measurements paired with reference images for learned reconstruction.
- fastMRI: Raw MRI k-space data with reference images for accelerated reconstruction; free registration and data agreement required.
Your first project
Download a subset of LoDoPaB-CT, reconstruct images by applying a small U-Net after filtered back-projection, and compare it with filtered back-projection alone using PSNR and SSIM.
Common questions
What is an inverse problem?
An inverse problem infers causes from observed effects: an image from scanner measurements, subsurface structure from seismic data or the state of the atmosphere from scattered observations. Many are ill-posed, so prior knowledge is needed to choose a sensible answer.
What is data assimilation?
Data assimilation combines a model forecast with new observations, weighted by their uncertainties, to estimate the current state of a system. Weather centres run it continuously to start every forecast.
Can deep learning invent detail in image reconstruction?
Yes. Learned priors can add plausible but false structures or remove real ones, especially on data unlike the training set. This is why reconstructions are checked for consistency with the measurements and evaluated on task-relevant features.
Is machine learning used in operational data assimilation?
It is being introduced for tasks such as observation operators, bias correction and model error estimation, alongside established variational and ensemble methods. ECMWF's machine-learned forecasts still start from initial conditions produced by physics-based data assimilation.
Courses for these skills
- Maths for Machine LearningConnect derivatives, vectors and physical objectives.26 published lessons · all planned lessons open
- Statistics and Probability for Machine LearningUnderstand noise and conditional uncertainty.41 published lessons · all planned lessons open
- Deep Learning with PyTorchStudy neural fitting and loss design.Syllabus available · lessons open as produced
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.