Science and engineering
Machine learning for radar signal processing
Radar engineers use machine learning to detect and classify objects in range-Doppler maps, to recognise human activities and gestures from micro-Doppler signatures, and to fuse radar with cameras in automotive perception.
How machine learning is used in radar signal processing
Recognising gestures and activity
Small radars sense fine motion through micro-Doppler patterns. Models trained on these signals recognise gestures, presence and activities without a camera.
Real exampleSoli radar in Pixel 4 (Google Research)Google described the Soli radar system behind Motion Sense on Pixel 4 and the machine learning algorithms used to recognise human activity from its radar data.
Fusing radar with cameras for driving
Radar measures range and velocity directly and works in poor weather, while cameras give appearance. Fusion networks combine them for more reliable 3D detection.
Real exampleCenterFusion (Nabati and Qi, 2020)A middle-fusion approach that associates radar detections with camera-based detections for 3D object detection, evaluated on the nuScenes dataset.
Classifying objects in radar point clouds
Automotive radars output sparse detections with position, velocity and strength. Models group and classify these points into cars, pedestrians and cyclists.
Real exampleRadarScenes (Schumann et al., 2021)An automotive radar dataset with point-wise annotations from more than four hours of driving, recorded by four series radar sensors, for developing machine learning radar perception.
Detecting objects in range-Doppler maps
Working on the processed radar tensor, rather than a sparse list of detections, keeps more information. Segmentation networks label objects directly in range, angle and Doppler.
Real exampleCARRADA dataset (Ouaknine et al., 2020)A dataset of synchronised camera and automotive radar recordings with range-angle-Doppler annotations for radar object detection and segmentation.
Can you learn machine learning for radar?
Yes. Radar engineers can add machine learning to their signal processing knowledge, and machine learning engineers can learn radar basics. The usual order is Python, signals and linear algebra, classical classifiers on radar features, then deep learning on range-Doppler maps and point clouds.
- PythonProcess raw radar data and build range-Doppler maps with NumPy.Learn it in Python
- Signals and linear algebraFourier transforms, filtering and matrix methods underpin range, Doppler and angle processing.Learn it in Maths
- Classical classifiersClassify targets from micro-Doppler or point-cloud features with tree models and SVMs.Learn it in Machine Learning
- Deep learningConvolutional networks on range-Doppler maps and point-cloud networks on detections.Learn it in Deep Learning
- Radar fundamentalsFMCW waveforms, CFAR detection, clutter, multipath and calibration.
Moving into machine learning for radar
If you already work in the field
Radar engineers know waveforms, processing chains and sensor limits. Add Python and PyTorch, then train a classifier on range-Doppler maps from an open dataset such as CARRADA or RadarScenes. Automotive, defence, consumer electronics and health sensing teams need this mix.
If you come from data science or ML
Machine learning practitioners should learn FMCW radar basics, range-Doppler-angle processing, clutter and multipath, and why radar data is sparse and noisy compared with images. A radar signal processing text and open automotive radar datasets are the usual route.
Roles that do this workRadar signal processing engineerPerception engineerMachine learning engineer, sensingRadar systems engineer
Tools you will use
| Tool | What you use it for |
|---|---|
| SciPy | Compute FFTs, filters and spectrograms from raw radar returns. |
| PyTorch | Train networks on range-Doppler maps, spectrograms and point clouds. |
| scikit-learn | Fit feature-based classifiers as transparent baselines. |
| nuScenes devkit | Load and visualise radar, camera and lidar data from the nuScenes dataset. |
Public data to practise on
- Radar signatures of human activities (Glasgow): Radar recordings of people walking, sitting, falling and other activities for micro-Doppler classification.
- CARRADA: Synchronised automotive radar and camera recordings with range-angle-Doppler annotations.
- nuScenes: Driving scenes with radar, camera and lidar plus 3D object labels; free registration required.
Your first project
Download the University of Glasgow radar human-activity dataset, classify activities from micro-Doppler spectrograms with a small convolutional network, and compare it with handcrafted spectrogram features and a random forest using accuracy on held-out participants.
Common questions
How is deep learning used in radar?
For detecting and classifying objects, recognising gestures and activities from micro-Doppler signatures, suppressing interference and fusing radar with cameras or lidar.
What is a micro-Doppler signature?
It is the pattern of small Doppler shifts caused by moving parts of a target, such as swinging arms and legs or rotor blades. Time-frequency plots of these shifts are distinctive enough to classify activities and targets.
Why is radar used alongside cameras in cars?
Radar measures range and velocity directly and works in rain, fog and darkness, while cameras capture rich appearance. Fusing them gives more robust perception than either alone.
Is there public radar data to practise with?
Yes. Automotive datasets such as nuScenes, RadarScenes and CARRADA include radar measurements with annotations.
Courses for these skills
- Deep Learning with PyTorchStudy CNNs and object detection in the Computer Vision syllabus.Syllabus available · lessons open as produced
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnKeep annotation coordinates and image identifiers aligned.23 published lessons · all planned lessons open
- Practical Machine Learning in PythonCompare feature-based classifiers.51 published lessons · more lessons in production
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.