← Adversarial Math From First Principles

What Counts as Proof?

Five things people call proof, from algebra to kernel-checked theorems: the evidence-status ladder and how the engine refuses to be fooled by a compiling file with sorry.

Here are four artifacts. A reader might call each of them “proved”:

  1. A paragraph of algebra that expands the squares.
  2. A SymPy call that simplifies a residual to 0.
  3. A Z3 run that returns unsat on the negation.
  4. A Lean file that compiles.

The fourth is the dangerous one. This file compiles:

import Mathlib

theorem intentionallyIncomplete (x : ℝ) : x = x := by
  sorry

Lean accepts it with a warning, because sorry inserts an axiom (sorryAx) that proves anything. The repository keeps this file as formal/lean/fixtures/Incomplete.lean for exactly that reason: its header says the adapter “must reject this placeholder even if Lean permits it with a warning.” It compiled is not it was proved.

This chapter separates the things people call “a proof” and gives each its own status. It builds on the claims of Chapter 1, and it is the backbone of the tool chapters (Chapters 9 to 14).

The ladder of statuses

The engine’s evidence policy and models define these statuses:

Status What it requires What it does not imply
formally_verified An actual proof-kernel command accepted the exact recorded proposition and assumptions That the encoding matches another prover, or physical reality
computation_verified_within_stated_domain A real symbolic or solver computation established the encoded result in its declared domain A kernel proof, or empirical validation
refuted_by_valid_counterexample A concrete assignment violates a universal claim and was re-evaluated That a broader theory is false
inconclusive Attempts exist but neither established nor refuted the claim Support, by default
unsupported The required encoding or proof is absent or uses prohibited gaps Falsehood
not_evaluated No qualifying execution occurred Anything about the claim

These are not points on a “strength score”. They answer different questions.

What each kind of result is

An informal proof is an argument a human can follow. The expansion in Chapter 20 is one. The engine records no status for it, because nobody ran anything. Its value is as the thing a human reads and checks.

A proof term checked by a kernel is a small program in a logic whose checker is simple enough to trust. Lean’s and Rocq’s kernels are the ones used here. When the engine says formally_verified, it means the kernel accepted exactly this proposition under exactly these axioms. Tactics such as ring, nlinarith and field_simplify are untrusted helpers that produce the term. If they produce something wrong, the kernel rejects it.

A computation is a result produced by a program and not checked by a kernel. SymPy reducing $\left(1/\sqrt{1-v^2}\right)^2(1-v^2) - 1$ to $0$ is computational evidence within an encoded domain, and the status says exactly that: computation_verified_within_stated_domain.

A solver result is similar. The evidence policy says to treat Z3 unknown as inconclusive, and to read unsat only as unsatisfiability of the recorded negation and assumptions. The Z3 run on the algebraic theorem returned unsat for the negation under $\gamma^2(1-v^2)=1$. That supports the encoded statement. It is a different thing from a kernel proof.

A certificate is a witness that can be checked independently. The engine’s one fully developed case is a counterexample (Chapter 5), which is re-checked by substitution. This edition found no proof-certificate check for unsat results, so that question stays open.

The same distinctions, tool by tool:

Tool Role in this book What it can establish What it does not establish
SymPy Symbolic computation A residual or identity within the encoded expression A kernel theorem, or that a physical side condition is enforced
Z3 SMT solver unsat for the encoded negation; a sat model or witness A checked proof certificate, arbitrary nonlinear arithmetic, or a kernel proof
Lean 4 Proof kernel Acceptance of the exact proposition under its axioms That the formal statement matches the prose, cross-kernel equivalence, or physical truth
Rocq Proof kernel Acceptance of a separately written proposition Semantic equivalence with Lean, or physical truth
Isabelle/HOL Candidate kernel and countermodel finder A Docker build of the repaired candidate copy and a finite Nitpick countermodel (Chapter 13); no engine result An engine EvidenceResult or accepted source change; a Nitpick model is not a proof of the algebraic claim
SageMath Candidate computation An illustrative Docker computation (Chapter 14); no engine result A claim-bound EvidenceResult or formal proof

How the engine refuses to be fooled

These three guards are taught here once; later chapters refer back to them.

  1. Placeholder screening. Before running a kernel, the external proof adapter scans the source for prohibited tokens. For Lean: sorry, admit, a leading axiom. For Rocq: Admitted., admit, Axiom. A match yields unsupported, not success.
  2. Evidence-inflation guard. An EvidenceResult will not even construct with a strong status unless the execution mode is actual and the execution status is succeeded. A simulated run, a timeout or a missing tool cannot carry formally_verified.
  3. Axiom audit. scripts/ci_formal_audit.py records each theorem’s axiom dependencies with Lean’s #print axioms and Rocq’s Print Assumptions, and enforces the exact expected sets for the pinned versions. sorryAx, Admitted or an unexpected axiom fails the audit.

Guards against false success need one more thing: proof that the kernel can say no. formal/lean/fixtures/KernelRejected.lean states something false ((1 : Nat) = 2 := by rfl) with no placeholder and no added axiom, and the real Lean kernel rejects it. A pipeline that has never seen a real rejection has not shown that it can reject.

What “proved” rests on

“Proved” always means “proved from something”. Here are the logical axioms the case-study proofs depend on, from the R2 audit:

Theorem Lean Rocq
algebraic propext, Classical.choice, Quot.sound sig_forall_dec, functional_extensionality_dep
gamma bridge propext, Classical.choice, Quot.sound sig_forall_dec, sig_not_dec, functional_extensionality_dep

These are standard classical-reals and extensionality axioms. They are few, but they are not zero.

What this establishes

The chapter introduces no new claim; it uses the Lorentz claims of Chapter 20. Their strongest statuses in the accepted report make the chapter’s point:

Claim Strongest status in the accepted report
lorentz-interval-invariance-1d-algebraic@1.0.0 formally_verified (Lean and Rocq)
lorentz-interval-invariance-1d-physical@1.0.0 computation_verified_within_stated_domain (SymPy)

The second row is the lesson. At version 1.0.0 the physical claim has a very convincing computation and no proof, and the report does not round it up.

Three further limits apply to every status on the ladder:

  • A kernel accepting a proof is not a person understanding it. For MATH01 the record of what a human inspected is empty.
  • Two kernels accepting similarly named theorems are two checks on two propositions. Whether the propositions are the same is a separate obligation, and the project has not met it (semantic_equivalence_not_established). Chapter 15 takes it up.
  • A proof establishes a conditional: from these axioms, this statement. It does not establish that the statement is the one you meant (Chapter 1) or that nature obeys it (Chapter 4).

Try it yourself

uv run python scripts/ci_negative_controls.py
uv run python scripts/ci_kernel_rejection.py
uv run python scripts/ci_formal_audit.py

The first checks the false-transform refutations and placeholder rejection. The second asks the real kernels to reject false sources that contain no placeholder. The third audits theorem axioms. The last two need Lean and Rocq.

Seeded errors, each of which the repository already tests:

  • Put sorry in a proof: expect unsupported.
  • Replace the algebraic theorem’s hypothesis with True: expect a kernel rejection.
  • Use a mock executable that always exits 0: expect simulated, which cannot carry strong evidence.

Open questions.

  1. The axiom audit enforces exact sets for pinned versions. What happens when Mathlib is upgraded and a proof legitimately picks up a different standard axiom?
  2. Is Classical.choice acceptable for every claim the book will make? Some readers will want constructive content.
  3. What would a certificate-checked unsat look like, and should it earn a status above computation_verified_within_stated_domain?

Sources

These are the general references for kernels, tactics and sorry; no specific page is cited for the behaviour described above. In the repository: the evidence policy (docs/methodology/evidence-policy.md) and the adapter rules (src/math_engine/adapters/external.py).