Example

Put every AI proposal through an exact check

Confident proposals, some right and some wrong, accepted only when an exact check agrees; three 'distinct proofs' that are one text.

In this book, AI proposed: statements, proofs, transcriptions, prose. Nothing it produced counted until a check that could have rejected it agreed. This program shows the rule with a fixed list of proposals standing in for an assistant’s: confident and wrong, confident and right, and the same text counted three times.

Run it

Save this as proposer.py and run it with Python 3.11 or later. It uses the standard library only.

"""AI as proposer, not authority: every proposal goes through an exact check (standard library only).

The 'proposals' are a fixed list standing in for an assistant's suggestions, written to
show the failure modes the book records: confident and wrong, right, and duplicated.
"""
import hashlib
import re

PROPOSALS = [
    ("F5 = 2^32 + 1 is divisible by 641", lambda: (2 ** 32 + 1) % 641 == 0),
    ("F5 = 2^32 + 1 is divisible by 643", lambda: (2 ** 32 + 1) % 643 == 0),
    ("F6 = 2^64 + 1 = 274177 × 67280421310721", lambda: 2 ** 64 + 1 == 274177 * 67280421310721),
    ("65537 = 2^16 + 1 is composite", lambda: any(65537 % d == 0 for d in range(2, 257))),
    ("every Fermat number up to F4 is prime",
     lambda: all(all((2 ** 2 ** n + 1) % d for d in range(2, int((2 ** 2 ** n + 1) ** 0.5) + 1)) for n in range(5))),
]

print("Each proposal is accepted only if the exact check agrees:")
for text, check in PROPOSALS:
    print(f"  {'ACCEPT' if check() else 'reject'}  {text}")

print("\nMeasuring the wrong thing: three 'distinct proofs' that are one text")
proofs = [
    "theorem t : 0 + n = n := Nat.zero_add n",
    "theorem t : 0 + n = n :=  Nat.zero_add n",
    "theorem t :\n  0 + n = n := Nat.zero_add n",
]
norm = lambda s: re.sub(r"\s+", " ", s).strip()
raw = {hashlib.sha256(p.encode()).hexdigest()[:12] for p in proofs}
canonical = {hashlib.sha256(norm(p).encode()).hexdigest()[:12] for p in proofs}
print(f"  distinct by raw hash: {len(raw)}; distinct after collapsing whitespace: {len(canonical)}")
print("\nThe rule: a proposal is evidence only after a check that could have rejected it.")

Expected outcome

Each proposal is accepted only if the exact check agrees:
  ACCEPT  F5 = 2^32 + 1 is divisible by 641
  reject  F5 = 2^32 + 1 is divisible by 643
  ACCEPT  F6 = 2^64 + 1 = 274177 × 67280421310721
  reject  65537 = 2^16 + 1 is composite
  ACCEPT  every Fermat number up to F4 is prime

Measuring the wrong thing: three 'distinct proofs' that are one text
  distinct by raw hash: 3; distinct after collapsing whitespace: 1

The rule: a proposal is evidence only after a check that could have rejected it.

Two of the five proposals are rejected by exact arithmetic, however confidently stated: 643 does not divide F₅, and 65537 is prime. The others are accepted only because the check agrees. The “three proofs” collapse to one once whitespace is ignored: measuring the wrong thing is a failure of the counting, not of any proof.

Mechanism and limitations

Documented: Chapter 27 of the book sets out the mathematics; this program is a self-contained demonstration of it, not one of the laboratory’s recorded investigations.

Observed: the output above, produced by running the program exactly as shown.

Not shown:

  • The proposals are a fixed list written for this example, not output from an AI system.
  • An exact check accepts a proposal; it does not show that the proposal answers the question that was asked.

Related worked example: Chapter 6 .

Challenge this result

Copy this into the AI assistant of your choice, or work through it yourself. Check any finding against the recorded evidence.

Examine the proposals and the checks. For each accepted proposal, say what
the check establishes and what it does not. Write a proposal that would pass
its check but still mislead a reader, and propose a second check that would
catch it.

Chapter: https://programmer.ie/books/math/27-chapter/
Example: https://programmer.ie/examples/math/27-chapter/
Program: https://github.com/ernanhughes/math/blob/main/notebooks/math/programs/27-ai-proposer.py
Distinguish what the record shows from what you infer. If you cannot open a link, ask me for the material; do not invent it.

Canonical source and the program .

Understand this example

Copy this prompt into your AI tool. No code runs here.

Apply this example

Copy this prompt into your AI tool. No code runs here.