Health and life sciences
Machine learning for neural decoding
Neural decoding uses machine learning to predict what a subject sees, intends or does from recorded brain activity. Common tasks include classifying imagined movements from EEG, decoding stimuli from spiking neurons, and reading out position or velocity for brain-computer interfaces.
How machine learning is used in neural decoding
Decoding attempted movement into text
Recurrent networks map neural activity recorded during attempted movements onto characters or words in real time. The decoder is trained on labelled attempts and then run online while the participant uses it.
Real exampleHandwriting brain-computer interface (Willett et al., 2021)An intracortical interface decoded attempted handwriting from motor cortex with a recurrent neural network, and the study participant typed at 90 characters per minute with 94.1% raw accuracy online.
Decoding attempted speech
Spiking activity recorded during attempted speech is decoded into phonemes and then into words with a language model. Vocabulary size and word error rate are the measures that matter.
Real exampleSpeech neuroprosthesis (Willett et al., 2023)Decoded attempted speech from intracortical microelectrode arrays at 62 words per minute, with a 9.1% word error rate on a 50-word vocabulary and 23.8% on a 125,000-word vocabulary for the study participant.
Classifying EEG signals
Compact convolutional networks classify non-invasive EEG for interface paradigms such as evoked potentials and motor rhythms. They are attractive because EEG datasets are small and noisy.
Real exampleEEGNet (Lawhern et al., 2018)A compact convolutional network compared against reference algorithms across four brain-computer interface paradigms, which generalised across paradigms better than the references when training data was limited.
Identifying what someone is looking at
Encoding models first learn how each voxel responds to image features, then decoding inverts that model to identify which image was seen. This separates the model of the brain from the decoding step.
Real exampleIdentifying natural images from brain activity (Kay et al., 2008)Built receptive field models relating visual stimuli to fMRI activity in early visual areas and used them to identify which specific image an observer saw from a large set of novel images.
Extracting population dynamics
Latent variable models summarise many neurons into a few time-varying factors on single trials, and those factors are often easier to decode than raw spike counts.
Real exampleLFADS (Pandarinath et al., 2018)A deep learning method that infers latent dynamics from single-trial spiking data and, on macaque and human motor cortical datasets, predicted observed behavioural variables and stitched together non-overlapping recording sessions.
Can you learn machine learning for neural decoding?
Yes, and you can start on public recordings without a laboratory of your own. Learn Python and signal handling first, then statistics and regularised linear models, which are strong baselines on noisy neural data, then deep learning for sequence models. Neuroscience background helps you choose sensible labels and splits.
- Python and signal handlingLoad recordings with MNE-Python or PyNWB, filter them, and cut trials into epochs aligned to events.Learn it in Python
- Statistics for noisy signalsLearn cross-validation, chance levels and permutation tests, because decoding accuracies need a baseline to mean anything.Learn it in Statistics
- Regularised linear decodersFit ridge regression and linear classifiers on features from each trial, and treat them as the benchmark for anything more complex.Learn it in Machine Learning
- Sequence and latent variable modelsMove to convolutional and recurrent networks once the linear baseline is in place and you have enough trials.Learn it in Deep Learning
- Neural data conventionsLearn how experiments are structured, what each recording modality measures and why held-out runs or sessions are the honest test.
Moving into neural decoding work
If you already work in the field
Neuroscientists already have recordings, task structure and the judgement to spot an artefact. Add Python and model evaluation, then decode a variable you already analyse and compare the decoder with your standard analysis on held-out sessions.
If you come from data science or ML
Machine learning practitioners should learn how neural data is recorded, how much it drifts between sessions and why trial counts are small. Start with public datasets, keep linear baselines, and work with an experimenter who can tell you whether a result is plausible.
Roles that do this workComputational neuroscientistBrain-computer interface engineerNeural data scientistResearch software engineer, neuroscience
Tools you will use
| Tool | What you use it for |
|---|---|
| MNE-Python | Load, filter and epoch EEG and MEG data, then decode with built-in CSP and scikit-learn pipelines. |
| scikit-learn | Fit linear decoders and run cross-validation that respects runs and subjects. |
| pynapple | Align spike times, behaviour and events in time before building decoders. |
| PyNWB | Read Neurodata Without Borders files, the standard format for shared neurophysiology data. |
| CEBRA | Learn low-dimensional embeddings of neural activity that are consistent with behaviour. |
Public data to practise on
- EEG Motor Movement/Imagery Dataset (PhysioNet): Openly downloadable EEG from people performing and imagining hand and foot movements, for motor imagery decoding.
- DANDI Archive: Shared neurophysiology recordings in NWB format, including spiking data with behaviour for decoding.
Your first project
Download the PhysioNet EEG Motor Movement/Imagery dataset, decode imagined left versus right hand movement with CSP features and linear discriminant analysis in MNE-Python, and compare it with chance using cross-validated accuracy.
Common questions
Do you need implanted recordings to work on decoding?
No. Much of the field uses non-invasive EEG, MEG and fMRI, and large public datasets exist for all of them. Implanted recordings come from a small number of clinical research studies.
Why do simple linear models do so well?
Neural datasets usually have few trials and high noise, which favours models with strong regularisation. Deep models help mainly when data is plentiful or the mapping is strongly nonlinear.
How should decoding accuracy be judged?
Against a chance level computed for your class balance, on held-out runs, sessions or subjects, and ideally with a permutation test. Within-run splits leak slow drift and inflate scores.
Does decoding a variable prove a brain region represents it?
No. Decoding shows the information is present in the signal you recorded. Causal claims need perturbation experiments, not better classifiers.
Courses for these skills
- Practical Machine Learning in PythonCompare feature-based classifiers.51 published lessons · more lessons in production
- Statistics and Probability for Machine LearningAssess dependence and the limits of an experimental result.41 published lessons · all planned lessons open
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnPrepare arrays and preserve sampling metadata.23 published lessons · all planned lessons open
- Deep Learning with PyTorchExplore sequence representations after a baseline.Syllabus available · lessons open as produced
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.