Maths 5.1Lesson outlineLesson 1 of 1 in Module 5

Floating Point and Numerical Stability

Know why a correct formula can give a wrong answer on a computer, and the standard fixes.

Video in production.

Learn

The words this lesson introduces, each in one line. The module’s glossary collects them all.

floating-point number: a sign, a significand and an exponent, ±1.f × 2ᵉ | 0.75 = 1.1₂ × 2⁻¹
significand: the binary digits of the number, 53 of them in float64 | 1.1₂ for 0.75
exponent: the power of two the significand is multiplied by | −1 for 0.75
rounding error: the difference between a number and the float stored for it | 0.1 is stored 5.55 × 10⁻¹⁸ too high
machine epsilon: the gap between 1 and the next float, 2.22 × 10⁻¹⁶ in float64 | np.finfo(float).eps
relative error: the error divided by the size of the number | at most 1.11 × 10⁻¹⁶ after rounding
cancellation: the loss of correct digits when two nearly equal numbers are subtracted | (1 + h)² − 1 at h = 10⁻¹²
overflow: a result larger than the largest float, stored as inf | np.exp(710)
inf: the float for infinity | np.exp(1000)
underflow: a result closer to zero than the smallest float, stored as 0.0 | np.exp(-746)
nan: "not a number", the result of an undefined operation such as inf / inf | np.exp(1000) / np.exp(1000)
log-sum-exp: log Σ e^(zᵢ), computed as m + log Σ e^(zᵢ − m) with m the largest zᵢ | 1001.3133 for [1000, 1001]
numerically stable: computed so that rounding, overflow and underflow do not change the answer | log_expit(-1000) → −1000.0