Day 18: Finite differences, step size, and floating-point effects

01 120 min

Finite differences are approximations

Agent-Assisted Engineering Check

Prerequisites

Complete the previous activity and use the listed recovery source if blocked.

Learning Objective

Implement forward, backward, and central finite differences from the derivative definition, keep h explicit, and compare them with an analytical answer rather than treating an approximation as exact.

Instructions

1. Write the three stencils with a symbolic h before inserting numbers: forward [f(a+h) - f(a)]/h, backward [f(a) - f(a-h)]/h, central [f(a+h) - f(a-h)]/(2h). Keep the sign of h visible.
2. Predict which stencil lands closest to f'(3) = 6, using Day 17's result that the forward quotient for x^2 at 3 is exactly 6 + h.
3. Compute f(3.1), f(3) and f(2.9) once, then reuse those three values in all three stencils.
4. Tabulate each estimate next to its signed error (estimate - 6).
5. Explain the pattern with a Taylor expansion: for x^2 the forward error is exactly +h and the backward error exactly -h, and in the central stencil these cancel; the next term would involve f''', which is zero for a quadratic.
6. Say which quantities are exact (f'(3), the function values) and which are approximations (the stencil outputs in general), and why the central result here is a special case, not a rule.

Time plan (120 min): reading 30 + exercise 55 + check 35. Reading: OpenStax Section 3.1 (the derivative as a limit of difference quotients) and the SciPy derivative page overview. Exercise: the three stencils, the error table and the Taylor explanation. Check: the mastery item at x=2, h=.01, unaided.

Practice & Prediction Log

For f(x)=x^2 at x=3 and h=.1, calculate forward [f(3+h)-f(3)]/h, backward [f(3)-f(3-h)]/h, and central [f(3+h)-f(3-h)]/(2h). Predict which is closest to f'(3)=6.

Mark as Completed

Independent Mastery Criterion

Unaided, calculate all three estimates for f(x)=x^2 at x=2 and h=.01, compare with 4, and state which quantities are exact versus approximate.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 4.0 attribution noncommercial share-alike

OpenStax Calculus Volume 1

Author: OpenStax, Rice University (Gilbert Strang, Edwin Herman, et al.) | Original Source Link ↗

Relevant Section / Range: Sections 2.1-2.3 (Functions, Limits, and the Derivative); Sections 3.1-3.2 (Derivative Rules)

Verified Version & Date: 3rd edition, verified 2026-10-30

Selection Rationale:

Open, systematic treatment of the limit definition and derivative rules with stable section locations.

Source Prerequisites:

Algebra, coordinate geometry, and function notation

Safe Fallback (Offline Resilience):

The authored derivations, tables, and exercises in these three Study Days are complete offline recovery material; use the linked section only for optional comparison.

Compared Candidate Sources & Selection Analysis (3)

MIT OpenCourseWare 18.01SC Single Variable Calculus ↗ compared_selected_for_visual_practice [CC BY-NC-SA 4.0]

Strong exercises and transcripts, but the noncommercial share-alike terms and course navigation make OpenStax the primary stable text.

Useful reference, but rights and redistribution terms are less suitable for embedded recovery material.

SciPy Numerical Differentiation Documentation ↗ compared_selected_for_numerics [BSD-3-Clause]

A primary API reference for numerical derivative step-size behavior; it complements rather than replaces the hand derivation.

linking BSD-3-Clause

SciPy numerical derivative reference

Author: SciPy Developers | Original Source Link ↗

Relevant Section / Range: derivative API: finite differences, step-size iteration, tolerances

Verified Version & Date: SciPy documentation, verified 2026-10-30

Selection Rationale:

Documents numerical differentiation as an approximation with finite step sizes and stopping criteria.

Source Prerequisites:

Python arithmetic and the definition of a derivative

Safe Fallback (Offline Resilience):

The local Day 18 tables and pure-Python snippets reproduce the relevant finite-difference behavior without SciPy.

Compared Candidate Sources & Selection Analysis (2)

NumPy: numpy.gradient ↗ compared_alternative [BSD 3-Clause License]

Array-oriented gradient estimation on sampled grids; the SciPy derivative reference matches the pointwise finite-difference helper the exercises implement.

SymPy Calculus tutorial ↗ compared_alternative [BSD License]

Symbolic rather than numerical differentiation; useful for exact cross-checks but not the finite-difference practice target.

02 120 min

Step size: truncation versus roundoff

Independent Mastery Check (no AI assistance).

Prerequisites

Complete the previous activity and use the listed recovery source if blocked.

Learning Objective

Explain why smaller h is not universally better: truncation error often decreases with h, while floating-point cancellation and rounding can grow as nearby function values become indistinguishable.

Instructions

1. Predict before computing: the central-difference truncation error shrinks roughly like h^2, while the roundoff error grows roughly like eps/h (eps ≈ 2.2e-16 in double precision). Write down where you expect the two to cross.
2. Compute the central estimate (sin(1+h) - sin(1-h))/(2h) for h = .1, 1e-3, 1e-5, 1e-8 and 1e-12, printing at least 12 significant digits.
3. Measure each absolute error against math.cos(1) at working precision, not against the 10-decimal display 0.5403023059.
4. Mark the row where the error stops decreasing and starts growing (the turnover).
5. Explain the turnover in two sentences: one for truncation, one for cancellation and roundoff.
6. State explicitly that this table shows one function at one point in one precision; it does not give a universal best h.

Time plan (120 min): reading 25 + exercise 60 + check 35. Reading: the SciPy derivative page (step size, tolerances) and your Activity 1 notes. Exercise: the five-row table with errors and the turnover. Check: the mastery item, reproducing three rows unaided and explaining why 1e-12 is worse than 1e-5.

Practice & Prediction Log

Use central differences for sin(1), whose analytical derivative is cos(1)≈0.5403023059 (10 displayed decimals; measure errors against cos(1) at working precision). Compare h=.1, 1e-3, 1e-5, 1e-8, and 1e-12. Explain the best region rather than selecting the smallest h by habit.

Mark as Completed

Independent Mastery Criterion

Without assistance, reproduce at least three table rows with a calculator or Python, compute absolute error against cos(1), and explain why h=1e-12 is worse than h=1e-5 here.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 4.0 attribution noncommercial share-alike

OpenStax Calculus Volume 1

Author: OpenStax, Rice University (Gilbert Strang, Edwin Herman, et al.) | Original Source Link ↗

Relevant Section / Range: Sections 2.1-2.3 (Functions, Limits, and the Derivative); Sections 3.1-3.2 (Derivative Rules)

Verified Version & Date: 3rd edition, verified 2026-10-30

Selection Rationale:

Open, systematic treatment of the limit definition and derivative rules with stable section locations.

Source Prerequisites:

Algebra, coordinate geometry, and function notation

Safe Fallback (Offline Resilience):

The authored derivations, tables, and exercises in these three Study Days are complete offline recovery material; use the linked section only for optional comparison.

Compared Candidate Sources & Selection Analysis (3)

MIT OpenCourseWare 18.01SC Single Variable Calculus ↗ compared_selected_for_visual_practice [CC BY-NC-SA 4.0]

Strong exercises and transcripts, but the noncommercial share-alike terms and course navigation make OpenStax the primary stable text.

Useful reference, but rights and redistribution terms are less suitable for embedded recovery material.

SciPy Numerical Differentiation Documentation ↗ compared_selected_for_numerics [BSD-3-Clause]

A primary API reference for numerical derivative step-size behavior; it complements rather than replaces the hand derivation.

linking BSD-3-Clause

SciPy numerical derivative reference

Author: SciPy Developers | Original Source Link ↗

Relevant Section / Range: derivative API: finite differences, step-size iteration, tolerances

Verified Version & Date: SciPy documentation, verified 2026-10-30

Selection Rationale:

Documents numerical differentiation as an approximation with finite step sizes and stopping criteria.

Source Prerequisites:

Python arithmetic and the definition of a derivative

Safe Fallback (Offline Resilience):

The local Day 18 tables and pure-Python snippets reproduce the relevant finite-difference behavior without SciPy.

Compared Candidate Sources & Selection Analysis (2)

NumPy: numpy.gradient ↗ compared_alternative [BSD 3-Clause License]

Array-oriented gradient estimation on sampled grids; the SciPy derivative reference matches the pointwise finite-difference helper the exercises implement.

SymPy Calculus tutorial ↗ compared_alternative [BSD License]

Symbolic rather than numerical differentiation; useful for exact cross-checks but not the finite-difference practice target.

03 120 min

Numerical experiment and failure analysis

Agent-Assisted Engineering Check

Prerequisites

Complete the previous activity and use the listed recovery source if blocked.

Learning Objective

Run a reproducible pure-Python finite-difference experiment, compare forward and central errors, and diagnose cancellation using sqrt(1+h)-sqrt(1) as h becomes tiny.

Instructions

1. Start the script by printing sys.version so the run can be reproduced.
2. Keep the oracle separate: compute math.cos(1) once and never derive it from a finite difference.
3. Loop over h = .1, 1e-3, 1e-5, 1e-8, 1e-12 and print h, the forward error and the central error for sin(1) in scientific notation.
4. Add the cancellation case: the forward estimate (sqrt(1+h) - sqrt(1))/h at h = 1e-8, 1e-12 and 1e-16. Before running it, predict what 1 + 1e-16 evaluates to in double precision and what that does to the numerator.
5. Keep every row, including the unstable ones; an experiment that hides bad rows is not evidence.
6. For each regime write one diagnosis line: truncation-dominated, cancellation/roundoff-dominated, or total loss of significance.

Time plan (120 min): reading 20 + exercise 70 + check 30. Reading: the hint and your Activity 2 table. Exercise: write and run the script and capture its full output. Check: compare your table and diagnoses with the reference, then rerun with different formatting if a row disagrees.

Practice & Prediction Log

Write a short script that prints h, forward error, and central error for sin(1). Add h=1e-16 for the square-root example and predict whether the numerator remains distinguishable in floating point.

Mark as Completed

Independent Mastery Criterion

Independent work must include executable code, a captured table with at least four h values, and a written diagnosis distinguishing truncation error from cancellation/roundoff.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 4.0 attribution noncommercial share-alike

OpenStax Calculus Volume 1

Author: OpenStax, Rice University (Gilbert Strang, Edwin Herman, et al.) | Original Source Link ↗

Relevant Section / Range: Sections 2.1-2.3 (Functions, Limits, and the Derivative); Sections 3.1-3.2 (Derivative Rules)

Verified Version & Date: 3rd edition, verified 2026-10-30

Selection Rationale:

Open, systematic treatment of the limit definition and derivative rules with stable section locations.

Source Prerequisites:

Algebra, coordinate geometry, and function notation

Safe Fallback (Offline Resilience):

The authored derivations, tables, and exercises in these three Study Days are complete offline recovery material; use the linked section only for optional comparison.

Compared Candidate Sources & Selection Analysis (3)

MIT OpenCourseWare 18.01SC Single Variable Calculus ↗ compared_selected_for_visual_practice [CC BY-NC-SA 4.0]

Strong exercises and transcripts, but the noncommercial share-alike terms and course navigation make OpenStax the primary stable text.

Useful reference, but rights and redistribution terms are less suitable for embedded recovery material.

SciPy Numerical Differentiation Documentation ↗ compared_selected_for_numerics [BSD-3-Clause]

A primary API reference for numerical derivative step-size behavior; it complements rather than replaces the hand derivation.

linking BSD-3-Clause

SciPy numerical derivative reference

Author: SciPy Developers | Original Source Link ↗

Relevant Section / Range: derivative API: finite differences, step-size iteration, tolerances

Verified Version & Date: SciPy documentation, verified 2026-10-30

Selection Rationale:

Documents numerical differentiation as an approximation with finite step sizes and stopping criteria.

Source Prerequisites:

Python arithmetic and the definition of a derivative

Safe Fallback (Offline Resilience):

The local Day 18 tables and pure-Python snippets reproduce the relevant finite-difference behavior without SciPy.

Compared Candidate Sources & Selection Analysis (2)

NumPy: numpy.gradient ↗ compared_alternative [BSD 3-Clause License]

Array-oriented gradient estimation on sampled grids; the SciPy derivative reference matches the pointwise finite-difference helper the exercises implement.

SymPy Calculus tutorial ↗ compared_alternative [BSD License]

Symbolic rather than numerical differentiation; useful for exact cross-checks but not the finite-difference practice target.

04 120 min

Retrieval, review, and independent mastery

Independent Mastery Check (no AI assistance)

Prerequisites

Complete the previous activity and use the listed recovery source if blocked.

Learning Objective

Integrate analytical derivatives, nondifferentiability, finite differences, and numerical error into a defensible verification protocol for model-training mathematics.

Instructions

1. Close the book and your notes. Answer parts (1)-(6) of the prompt from memory, in order.
2. For (1) and (2), give the argument and not just the value: name the rule used for x^3, and give both one-sided slopes for |x| at 0.
3. For (3), define a protocol: a stencil, a set of h values spanning several orders of magnitude, and an acceptance rule (for example a relative-error tolerance that the estimates approach before roundoff takes over).
4. For (4), list the evidence fields you would record: function, point, dtype, h values, estimates, oracle, errors, and failures.
5. For (5), choose your own function, point and h, state them, and carry the central-difference calculation through to an error against the analytical derivative.
6. For (6), write the limitation sentence in your own words.
7. Mark your answers against the reference. For every miss, revisit the matching activity of Days 16-18 and redo an equivalent item unaided.

Time plan (120 min): reading 15 + exercise 60 + check 45. Reading: none; this is closed-book retrieval. Exercise: the six-part response and the worked calculation. Check: mark, revisit, and redo each missed item unaided.

Practice & Prediction Log

Closed book: (1) give the derivative of x^3 at 2; (2) state whether |x| is differentiable at 0 and why; (3) choose a finite-difference stencil and h policy for checking a black-box smooth function; (4) list what evidence you would record; (5) include a worked central-difference calculation on a function, point, and h you choose and state explicitly; (6) add an explicit statement that a finite-difference check is verification evidence, not a proof of differentiability or exactness.

Mark as Completed

Independent Mastery Criterion

Independent Mastery Check: submit the four-part response plus a worked central-difference calculation and an explicit statement that finite differences are verification evidence, not proof of differentiability or exactness.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 4.0 attribution noncommercial share-alike

OpenStax Calculus Volume 1

Author: OpenStax, Rice University (Gilbert Strang, Edwin Herman, et al.) | Original Source Link ↗

Relevant Section / Range: Sections 2.1-2.3 (Functions, Limits, and the Derivative); Sections 3.1-3.2 (Derivative Rules)

Verified Version & Date: 3rd edition, verified 2026-10-30

Selection Rationale:

Open, systematic treatment of the limit definition and derivative rules with stable section locations.

Source Prerequisites:

Algebra, coordinate geometry, and function notation

Safe Fallback (Offline Resilience):

The authored derivations, tables, and exercises in these three Study Days are complete offline recovery material; use the linked section only for optional comparison.

Compared Candidate Sources & Selection Analysis (3)

MIT OpenCourseWare 18.01SC Single Variable Calculus ↗ compared_selected_for_visual_practice [CC BY-NC-SA 4.0]

Strong exercises and transcripts, but the noncommercial share-alike terms and course navigation make OpenStax the primary stable text.

Useful reference, but rights and redistribution terms are less suitable for embedded recovery material.

SciPy Numerical Differentiation Documentation ↗ compared_selected_for_numerics [BSD-3-Clause]

A primary API reference for numerical derivative step-size behavior; it complements rather than replaces the hand derivation.

linking BSD-3-Clause

SciPy numerical derivative reference

Author: SciPy Developers | Original Source Link ↗

Relevant Section / Range: derivative API: finite differences, step-size iteration, tolerances

Verified Version & Date: SciPy documentation, verified 2026-10-30

Selection Rationale:

Documents numerical differentiation as an approximation with finite step sizes and stopping criteria.

Source Prerequisites:

Python arithmetic and the definition of a derivative

Safe Fallback (Offline Resilience):

The local Day 18 tables and pure-Python snippets reproduce the relevant finite-difference behavior without SciPy.

Compared Candidate Sources & Selection Analysis (2)

NumPy: numpy.gradient ↗ compared_alternative [BSD 3-Clause License]

Array-oriented gradient estimation on sampled grids; the SciPy derivative reference matches the pointwise finite-difference helper the exercises implement.

SymPy Calculus tutorial ↗ compared_alternative [BSD License]

Symbolic rather than numerical differentiation; useful for exact cross-checks but not the finite-difference practice target.