Data and modeling
Machine learning XOR problem
The XOR problem is the classic example of data that no single straight line can separate: the output is 1 when exactly one input is 1. A neural network with one hidden layer, or a hand-made product feature, solves it, which is why XOR shows why nonlinear models matter.
How the XOR problem is used to teach machine learning
Introducing feedforward neural networks
XOR is small enough to solve by hand, yet no single linear boundary separates its classes. Textbooks use it to show how a hidden layer transforms the inputs into a space where a linear output works.
Real exampleDeep Learning (Goodfellow, Bengio and Courville), section 6.1Chapter 6 opens with 'Example: Learning XOR', showing that a linear model fails and that a network with a learned hidden representation solves the task.
Motivating backpropagation
Networks with hidden units can represent XOR, but someone has to find their weights. Backpropagation computes how each weight affects the error, which made training such networks practical.
Real exampleLearning representations by back-propagating errors (Rumelhart, Hinton and Williams, Nature 1986)The paper describes back-propagation, a procedure that adjusts connection weights so that hidden units come to represent important features of the task.
Explaining feature crosses
Multiplying two features, x1 times x2, gives a linear model the extra column it needs to separate XOR-shaped data. Neural networks learn such combinations automatically instead of needing them by hand.
Real exampleGoogle Machine Learning Crash Course: Neural networksThe module shows a nonlinear classification problem that a feature cross x1x2 makes separable, and explains that neural networks are designed to find such nonlinear patterns automatically.
Feature interactions in production models
The XOR lesson, that some outcomes depend on combinations of features, shapes real systems. Recommenders combine explicit cross-product features with learned embeddings to capture interactions.
Real exampleWide & Deep Learning for Recommender Systems (Google, 2016)The paper combines a wide linear model with cross-product feature transformations and a deep neural network, and evaluates it on Google Play app recommendations.
Can you learn the XOR problem and neural networks?
Yes. XOR needs only four data points, so you can work it through on paper and in a few lines of NumPy. Learn vectors and matrix multiplication, then logistic regression, then a network with one hidden layer trained by gradient descent.
- Python and NumPyWrite the XOR truth table as arrays and compute weighted sums with NumPy.Learn it in Python
- Vectors, matrices and gradientsLearn the dot product, matrix multiplication and derivatives, which are all the maths a small network needs.Learn it in Maths
- Linear classifiersFit logistic regression and plot its straight-line boundary to see exactly why it cannot separate XOR.Learn it in Machine Learning
- From logistic regression to a neural networkAdd a hidden layer with a nonlinear activation, train it with backpropagation, and plot the learned boundary.Learn it in Deep Learning
Where to go after the XOR problem
If you already work in the field
If you met XOR in a logic, electronics or maths course, you already know the truth table. Code it in NumPy, fit logistic regression to see it fail, and train a two-layer network to see it succeed.
If you come from data science or ML
If you already use neural networks, revisit XOR to build intuition about hidden representations and activation functions, then move to noisy nonlinear data such as make_moons and to real feature-interaction problems.
Roles that do this workMachine learning engineerDeep learning engineerResearch scientistData scientist
Tools you will use
| Tool | What you use it for |
|---|---|
| NumPy | Build the XOR truth table and compute a hidden layer by hand. |
| scikit-learn MLPClassifier | Fit a small multilayer perceptron and compare it with logistic regression. |
| PyTorch | Write the two-layer network and training loop yourself to see backpropagation at work. |
| TensorFlow Playground | Train XOR-style networks in the browser and watch hidden units form the boundary. |
Public data to practise on
- scikit-learn make_moons: Generates two interleaving half-circles, a noisy nonlinear dataset for testing decision boundaries.
- scikit-learn make_circles: Generates one ring inside another, which no straight line can separate.
Your first project
Generate the four XOR points and a make_moons sample in scikit-learn, fit logistic regression and an MLPClassifier with one hidden layer, and compare their accuracy on a held-out moons sample.
Common questions
Why can't a perceptron solve XOR?
A single-layer perceptron draws one straight line, and the XOR classes sit on opposite corners of a square, so no single line separates them. A hidden layer or an added product feature fixes this.
How many hidden neurons are needed to solve XOR?
Two hidden units with a nonlinear activation are enough in principle. In practice, training can get stuck with only two, so a few more units or several random starts make it reliable.
Who showed the XOR limitation?
Marvin Minsky and Seymour Papert analysed the limits of single-layer perceptrons, including problems like XOR, in their 1969 book Perceptrons. Multi-layer networks trained with backpropagation later overcame it.
Is the XOR problem still relevant?
Yes, as a teaching example and as the simplest case of a feature interaction. Many real problems depend on combinations of inputs that a linear model cannot capture.
Courses for these skills
- Maths for Machine LearningWork with norms, dot products and vector geometry.26 published lessons · all planned lessons open
- Deep Learning with PyTorchStudy CNNs, representations and image-model evaluation.Syllabus available · lessons open as produced
- Generative AI: LLMs, RAG and AgentsConnect embeddings and retrieval to language-model applications.Syllabus available · lessons open as produced
- Python for AI and Machine LearningBuild the basic programming skills used to handle image records.33 published lessons · all planned lessons open
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.