Uncertainty Without Confidence Theatre

Imagine a dashboard that says “Algebraic Lorentz theorem: 94% verified.” What would the number mean? That 94% of tools agreed? That a model is 94% sure? That 94% of the proof is done? Each reading is wrong for this kind of claim, because a theorem is either checked or it is not. A number invites the reader to treat a missing check as a small deduction.

The design record (ADR 0001) rejects the idea outright. It lists “majority voting among tools” as a rejected alternative, because “a mistranslation can make every tool agree on the wrong proposition.”

So the engine reports what was established and what was not, with no probability, no confidence score and no tool vote. This chapter explains the vocabulary it uses instead, the precedence rules that combine results, and one case where the same claim gets different conclusions because different tools ran. It builds on Chapters 1 to 3 and Chapter 5.

Three statuses people confuse

There are eight evidence statuses (Chapter 3 has the full ladder). These three are the ones most often misread:

A missing tool gives not_installed and not_evaluated, never a pass. A mock run is simulated and cannot carry a strong status. A Z3 unknown is inconclusive. None of these is a small version of “verified”. Each is a different statement about what happened.

Precedence, not averaging

assess_claim considers only results that match the claim’s ID, version and fingerprint, and only those with actual, successful execution. Then it applies these rules in order:

1. kernel-verified and refuted together: inconclusive (a conflict that needs an audit);
2. a validated counterexample: refuted_by_valid_counterexample;
3. a kernel acceptance: formally_verified;
4. a computation: computation_verified_within_stated_domain;
5. some attempt: inconclusive;
6. nothing: not_evaluated.

This is an ordering of kinds of evidence, with no weights. A counterexample outranks a computation because it is a stronger kind of fact, not because it got more votes.

Bounded conclusions

On top of the claim status, the dossier layer (the P1 phase of the project) computes a BoundedConclusion. It is recomputed deterministically from the records, never written freehand:

Every conclusion also names the claim fingerprint, the supporting result and audit IDs, the required checks, the unresolved obligations, the limitations, and the prohibited inferences: things a reader must not conclude from it. A conclusion is a bounded statement, and the boundary travels with it.

source_claim_qualified is the highest rung, and the record is careful about it: it “never means ‘paper proved.’”

One claim, two conclusions

The most instructive data point in the repository comes from running the four MATH01 claims, built from the same code, on two machines:

On the local machine, Rocq was absent. The profile for the gamma and algebraic claims declares a Rocq check as required, so with Rocq missing the required-check rule (priority 4) gives incomplete. That is not a weakness in the claim. It is an honest report of the evidence that exists on that machine. The hosted run had both kernels and reached the higher rungs. Nothing was rounded up locally, and the notebook displays whichever bundle it was given without upgrading it.

Notice what is absent: any percentage. “Incomplete” is a concrete, testable state. A named required check did not run.

These are historical local and hosted bundles. The later Docker runs of Rocq, Isabelle and SageMath (Chapters 12 to 14) execute tools outside the engine and do not change these statuses after the fact.

What this establishes

This approach costs some verbosity and, at first, feels like understatement: “incomplete” is less satisfying than “mostly done”. The benefit is that every word in a conclusion can be traced to a record. When a reader asks “how sure?”, the engine’s answer is “here is what ran, here is what did not, and here is what that entitles you to say.”

Honest uncertainty, in this book, is a list of unmet obligations, not a probability. The unresolved_obligations field on every claim and every conclusion is where the uncertainty lives, written in words that name what would resolve it.

One limit carries over from Chapter 3. Where the dossier layer compares two tool bindings, it does a structural check and reports aligned, mismatch or similar. That is not equivalence, and the record carries semantic_equivalence = not_established alongside every conclusion, even source_claim_qualified.

Try it yourself

uv run math-engine case run experiments/relativity/math01.case.json \
  --clean --output .ci/math-val01-p2
uv run math-engine dossier build \
  experiments/relativity/dossiers/math01.dossier-plan.json \
  --case-bundle .ci/math-val01-p2 --clean --output .ci/math-val01-p3
uv run math-engine dossier verify .ci/math-val01-p3

The local result depends on which tools you have installed. If Rocq is absent, expect incomplete for the gamma and algebraic claims.

Seeded error. Delete the Rocq result from a stored bundle and rebuild the dossier. The gamma and algebraic conclusions should drop from formal_statement_verified and source_claim_qualified to incomplete. (The P3 tests do this kind of mutation; this edition did not run it.)

Open questions.

1. Is a deterministic precedence ladder really free of scoring? Choosing the order is a judgment. Defend counterexample above kernel, and conflicting_evidence above counterexample.
2. A reader may want to know how much independent evidence exists. Is “number of independent kernels” a legitimate fact to report, given that the engine refuses to turn it into a vote?
3. Where does the engine represent human uncertainty about a statement, such as the reviewer’s “plausible but not carefully checked” in Chapter 28? It is a recorded review outcome, not a status on the ladder.

Evidence record

The local and hosted mapping is taken from docs/research/MATH-VAL01-P4-evidence.md. The local P3 input gave gamma and algebraic incomplete. Hosted PR #9 run 40235370914 gave formal_statement_verified for gamma and source_claim_qualified for the algebraic project-owned source claim. The notebook (P4) copies P3’s conclusions exactly, and a notebook whose conclusion differs from P3 fails verification.

Sources

No external literature is cited. The book does not yet situate this approach against Bayesian or Dempster–Shafer treatments of uncertainty. In the repository: docs/methodology/evidence-policy.md, docs/architecture/theorem-communication.md (the priority table) and src/math_engine/models.py (assess_claim).