See Simpson's paradox in two published tables, exactly
Compare success rates as exact fractions: better in every group, yet equal or reversed overall.
Can one treatment be better in every group and not better overall? This program takes two published tables, Simpson’s own 1951 example and the kidney-stone comparison of Charig et al. (1986), and compares every rate as an exact fraction, so nothing depends on rounding.
Run it
Save this as simpson.py and run it with Python 3.11 or later. It uses the standard library only.
"""Simpson's paradox in two published tables, compared exactly as fractions.
Simpson (1951), Table 4: alive / total, untreated vs treated.
Charig et al. (1986), Tables I and II: success / total, percutaneous (B) vs all open
procedures (A); 'small' is mean stone diameter under 2 cm, 'large' is 2 cm or more,
or multiple stones. Standard library only: no floating point, nothing rounded.
"""
from fractions import Fraction as F
TABLES = {
"Simpson 1951": ("untreated", "treated",
{"men": {"untreated": (4, 7), "treated": (8, 13)},
"women": {"untreated": (2, 5), "treated": (12, 27)}}),
"kidney stones": ("B", "A",
{"small": {"B": (234, 270), "A": (81, 87)},
"large": {"B": (55, 80), "A": (192, 263)}}),
}
def analyse(lo, hi, groups):
for g, c in groups.items():
print(f" {g:6} {lo}: {F(*c[lo])!s:>7} {hi}: {F(*c[hi])!s:>7} {hi} better? {F(*c[hi]) > F(*c[lo])}")
total = {arm: F(sum(c[arm][0] for c in groups.values()), sum(c[arm][1] for c in groups.values()))
for arm in (lo, hi)}
pattern = "equal" if total[hi] == total[lo] else ("reversed" if total[hi] < total[lo] else "same way")
print(f" {'all':6} {lo}: {total[lo]!s:>7} {hi}: {total[hi]!s:>7} combined: {pattern}")
return pattern
for name, (lo, hi, groups) in TABLES.items():
print(name)
analyse(lo, hi, groups)
print()
# Failure control: one changed cell (7 treated survivors instead of 8) breaks the pattern.
lo, hi, groups = TABLES["Simpson 1951"]
changed = {**groups, "men": {"untreated": (4, 7), "treated": (7, 13)}}
print("control: Simpson's table with 7/13 instead of 8/13")
analyse(lo, hi, changed)Expected outcome
Simpson 1951
men untreated: 4/7 treated: 8/13 treated better? True
women untreated: 2/5 treated: 4/9 treated better? True
all untreated: 1/2 treated: 1/2 combined: equal
kidney stones
small B: 13/15 A: 27/29 A better? True
large B: 11/16 A: 192/263 A better? True
all B: 289/350 A: 39/50 combined: reversed
control: Simpson's table with 7/13 instead of 8/13
men untreated: 4/7 treated: 7/13 treated better? False
women untreated: 2/5 treated: 4/9 treated better? True
all untreated: 1/2 treated: 19/40 combined: reversedIn both tables the second arm wins in every group. Combined, Simpson’s association vanishes (both rates are 1/2), despite its common description as a reversal, while the kidney-stone table reverses. Changing one cell, as in the control, destroys the pattern. These are arithmetic facts about published counts; they say nothing about cause, significance or which table a decision should use.
What the laboratory checked
The recorded run 20261113T155438Z-ea1fcdd4 (evidence
) is the book’s result; the program above is a reader-sized demonstration of the same mathematics. Evidence: Fresh kernel check (Lean and Rocq) plus exact rational computation. Status: kernel_checked_correspondence_pending.
There are published 2×2×2 tables in which one arm has the higher success rate within every group, yet in the combined table the rates are equal (Simpson 1951) or reversed (kidney stones).
| Check | System | Recorded outcome |
|---|---|---|
lean |
Lean kernel | 7 of 7 declarations accepted and bound to their pinned statements |
rocq |
Rocq kernel | 7 of 7 declarations accepted and bound to their pinned statements |
compute |
Python (exact rational arithmetic) | exit 0, verdict patterns_as_published |
Failure controls: 8 of 8 rejected or refused as required (lean-wrong-cell, lean-claim-reversal, lean-stones-no-reversal, rocq-wrong-cell, rocq-claim-reversal, lean-binding-wrong-statement, rocq-binding-wrong-statement, compute-wrong-cell).
Correspondence P07-CORR-01 (does the formal statement say what the claim means?): pending the author’s approval.
Contract P07-SIMPSON-01 version 2, sha256 f768af571f3b762f…; result.json sha256 272f3b3c588e044d….
To reproduce the laboratory’s run, build the math-env image from a clone of the repository (Getting Started
) and run:
img=$(docker image inspect math-env --format '{{.Id}}')
docker run --rm --network none -e UV_OFFLINE=1 -e MATHENV_IMAGE_ID=$img -v "${PWD}:/source:ro" -v "${PWD}/out:/out" math-env uv run --offline math-engine proof simpson --out /out/proof-simpsonThis Docker form was tested for simpson (R-D10). To re-verify the recorded run itself without Docker, math-engine proof --verify experiments/investigations/p07-simpson/kernel-runs/2026-11-13-v2-pinned (R-V07).
Mechanism and limitations
Documented: the claim contract experiments/investigations/p07-simpson/investigation.json (version 2) pins the statements, checks and controls before anything runs; the recorded run directory holds result.json, the runner’s notebook and a SHA256SUMS manifest of every file.
Observed: the program’s output above, and the recorded run 20261113T155438Z-ea1fcdd4: Kernel-checked on the pinned toolchain. The correspondence between the claim and the formal statements awaits the author’s approval, so no formal finding is attached yet.
Not shown, as the laboratory’s own record states:
- Any causal conclusion, or which table a decision should rely on: that needs assumptions about how the groups were formed, which arithmetic cannot supply.
- Anything about statistical significance.
- That the source readings are correct beyond the AI transcription in P07-SOURCE-CHECK-01: an independent human check against the scans is still outstanding.
- Anything about the treatments’ clinical merits: the study was not randomised, and arm A pools three different operations.
Challenge this result
Copy this into the AI assistant of your choice, or work through it yourself. Check any finding against the recorded evidence.
Check the published counts in Simpson (1951), Table 4, and Charig et al.
(1986), Tables I and II, against the program's tables, and recompute every rate
exactly. Is Simpson's example correctly described as a vanishing association
rather than a reversal? Does pooling the three open procedures into one arm
change what the comparison means? Propose a statement that would need a causal
assumption, and say which.
Chapter: https://programmer.ie/books/math/08-chapter/
Example: https://programmer.ie/examples/math/08-chapter/
Recorded run: P07-SIMPSON-01 20261113T155438Z-ea1fcdd4, status kernel_checked_correspondence_pending; evidence: Fresh kernel check (Lean and Rocq) plus exact rational computation
Source and evidence: https://github.com/ernanhughes/math/tree/main/experiments/investigations/p07-simpson/kernel-runs/2026-11-13-v2-pinned
Distinguish what the record shows from what you infer. If you cannot open a link, ask me for the material; do not invent it.Understand this example
Copy this prompt into your AI tool. No code runs here.
Apply this example
Copy this prompt into your AI tool. No code runs here.