What is a neural network? From inputs to a trainable prediction
An artificial neural network is a model built from connected computational units with adjustable parameters. It transforms numerical inputs into outputs through layers. Training adjusts those parameters to reduce a chosen loss on examples; useful predictions still need to be tested on separate data.
SchoolWhool's current learning path covers Python, Data Tools, Maths, Statistics and classical Machine Learning. Open lessons' parts and key terms are free to read.
The parts of a neural network
A small forward-pass example
Imagine a unit with inputs 2 and 3, weights 0.5 and -1, and bias 1. Its weighted sum is 2 × 0.5 + 3 × (-1) + 1 = -1. A ReLU activation returns the larger of zero and that sum, so this unit outputs zero. These are illustrative values, not a trained model or a result from a SchoolWhool lesson.
A layer applies operations to multiple units, and the next layer receives their outputs. Write the input and output shape beside each operation. For a first exercise, being able to trace a batch through the model is more useful than adding layers you cannot explain.
Training: loss, backpropagation and optimisation
Run the forward pass
Produce predictions using the current parameters. Keep the task's target representation compatible with the output and loss.
Calculate the loss
Compare predictions with the training objective. The loss is a training signal, which may differ from the metric you use to explain the project.
Calculate gradients
Backpropagation computes derivatives through the connected operations using the chain rule. A gradient describes a local change, not proof that the model has found a globally best solution.
Apply an update
The optimiser changes parameters using gradients and its update rule. Repeat across training batches while monitoring an independent validation set.
Why an activation function matters
Composing linear transformations without nonlinearities still gives a linear transformation. Nonlinear activations make the layered representation more expressive. That capacity is useful only if the training data and objective reward a pattern that also holds beyond the training examples.
For a learning exercise, compare a linear model and a small network on the same task. Keep the split and preprocessing fixed. If you change the representation and model together, you cannot easily tell which change affected the evaluation.
How can a neural network go wrong?
Use a debugging checklist tied to evidence: Are the labels aligned with the inputs? Are the feature scales sensible? Does the model improve on a trivial baseline? Are training and validation behaviour moving apart? Inspect a small set of examples directly before trying a more elaborate architecture.
Record the settings and data split with every experiment. If a result looks unexpectedly strong, check for duplicate examples and features that reveal the target. A bigger model is not a remedy for an evaluation that already contains the answer.
What maths do you need for neural networks?
Learn vectors and matrix multiplication so layer shapes make sense; derivatives and the chain rule so gradients have a meaning; and probability and statistics so you can interpret outputs and evaluation. You can work through small numerical examples while building those skills.
SchoolWhool's Maths and Statistics tracks build that foundation, and Python and Data Tools help with implementation. The Machine Learning track supplies the broader modelling workflow. Specialist neural-network training is a next step beyond the published classical ML curriculum.
Where this guide takes you
- Use the examples to understand the task and its evaluation. They are suggested learning exercises, not measured results or production-ready systems.
- The linked SchoolWhool courses teach the foundations. Specialist deep learning, computer vision, NLP, generative AI and reinforcement learning tracks are not published.
- Open lessons' parts and key terms are free to read. A free account opens quizzes and saved progress, plus practice and other material in the free sample modules.
Questions
What is the difference between a neural network and deep learning?
Neural networks are a family of models. Deep learning uses networks with multiple layers of learned representations; not every neural network is described as deep.
What is an epoch?
An epoch is one pass through the training dataset. A batch is a group of examples used in a training step, so an epoch can contain many parameter updates.
Are backpropagation and gradient descent the same?
No. Backpropagation calculates gradients through the network. Gradient descent is an update method that uses gradients to change parameters.
Does a lower training loss mean a better model?
It means a better fit to that training objective on those examples. You need separate evaluation to judge whether the improvement transfers to new data.
Can I understand neural networks without advanced maths?
You can begin with small forward-pass examples and basic algebra. Add linear algebra, derivatives and probability as you study training and evaluation.
Sources
Technical references and further reading from the authors, educators and maintainers below. Read dates record when these references were checked; the examples and suggested exercises in this guide are SchoolWhool's own.
- Google: neural network conceptsRead 2026-10-14
- PyTorch: network layers and forward passRead 2026-10-14
Related reading
Maths for modellingWhat is deep learning?Overfitting vs underfittingPrerequisites for machine learningBuild the foundations for this subject
Follow SchoolWhool's learning path through Python, Data Tools, Maths, Statistics and Machine Learning. Start with the skills you need, read open lessons without an account, then sign in free for quizzes and saved progress. Free sample modules also open their practice and study material.
