PyTorch train, eval and no_grad: separate behavior from gradients

model.train() and model.eval() control the behavior of modules that distinguish training from evaluation. torch.no_grad() controls gradient recording. They address different questions: evaluation mode does not automatically disable autograd, and disabling gradients does not automatically disable dropout.

Confusing them can make validation noisy, waste inference memory, or change model state during evaluation. We will inspect a dropout module on an invented tensor, then check gradient behavior separately. The example runs on CPU and needs PyTorch; it does not download a model or training data.

Demonstrate the two independent controls

The tensor contains sixteen ones. Training dropout randomly masks entries and scales retained entries. Evaluation dropout passes its input through unchanged. The script checks those behaviors without relying on the exact sequence of mask values.

import torch
from torch import nn

torch.manual_seed(7)
x = torch.ones(16, requires_grad=True)
dropout = nn.Dropout(p=0.5)

dropout.train()
with torch.no_grad():
    masked = dropout(x)
print("training still masks without gradients:", bool((masked == 0).any()))
print("training result records gradients:", masked.requires_grad)

dropout.eval()
evaluated = dropout(x * 2)
assert torch.equal(evaluated, torch.full_like(x, 2))
print("eval records gradients:", evaluated.requires_grad)
evaluated.sum().backward()
assert torch.equal(x.grad, torch.full_like(x, 2))

with torch.inference_mode():
    prediction = dropout(x * 2)
print("inference result records gradients:", prediction.requires_grad)
assert torch.equal(prediction, evaluated.detach())

Expected output:

training still masks without gradients: True
training result records gradients: False
eval records gradients: True
inference result records gradients: False

The seed makes the training-mask demonstration reproducible in this checked environment. We inspect qualitative behavior rather than publishing a particular random mask as an API guarantee.

Understand what evaluation mode changes

Calling .eval() is equivalent to .train(False) and propagates the mode to child modules. Layers such as dropout and ordinary batch normalization use that flag. Many layers, including a plain linear layer, do not change their forward computation simply because evaluation mode is selected.

Batch normalization adds a state concern: with its usual running-statistics configuration, training updates those statistics while evaluation uses them. Running validation in training mode can therefore alter state using validation batches, even inside no_grad. Exact behavior depends on options such as tracking running statistics, so inspect the particular module rather than treating every normalization layer as identical.

The PyTorch Module reference documents the mode switch. Custom modules can also implement behavior based on their training flag.

Understand what gradient controls change

Gradient recording tracks operations needed for derivative calculations. Disabling it can reduce overhead when derivatives are not needed. It does not change a module's training flag or freeze every possible state update.

inference_mode disables additional bookkeeping beyond no_grad, but it also imposes restrictions on tensors used outside the context. Use it for an appropriate inference path; use a gradient-compatible path when downstream computation genuinely needs derivatives. The autograd notes distinguish these modes.

Conversely, evaluation with gradients can be intentional. Input-gradient analysis or certain optimization procedures may require derivatives while retaining evaluation behavior. Our backward() check demonstrates that the combination is possible; it does not recommend recording gradients for every prediction request.

Put the controls in a validation workflow

At the start of a training phase, call model.train(). For validation, switch to model.eval() and enter an appropriate gradient-disabled context. Accumulate metrics on the intended evaluation set. Restore training mode before the next training phase.

If validation is wrapped in a reusable helper, remember that its caller might already be evaluating. Capture the prior mode and restore it rather than always forcing training mode on return. More complex models with deliberately mixed child-module modes need a more careful restoration policy.

No mode switch corrects validation leakage from fitted preprocessing, repeated groups or future-derived features. Those belong to the experiment design, as shown in cross-validation for groups and time.

Diagnose a concrete symptom

If repeated validation passes give different predictions, inspect dropout mode, stochastic transforms and other random operations. If validation consumes unnecessary memory, inspect gradient recording and retained tensors. If later training changes unexpectedly after evaluation, inspect state updates and whether the previous mode was restored.

Then repeat the example with a simple batch-normalization layer and inspect its running mean before and after calls in each mode. Continue through Deep Learning resources and the convolution-shape walkthrough for another inspectable PyTorch boundary. The course syllabus labels which related lessons are available and which are planned.