# How Do You Measure a Hallucination?
# Book: Hallucination From First Principles
# Chapter URL: https://programmer.ie/books/hallucination-from-first-principles/04-chapter/
# Prose fingerprint (sha256, normalized): 9347edf9fe23270921588a6236daeb0ebf4aac7c0fc763bbabd4b292b15eaff4
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

The first three chapters deliberately avoided building a detector.
Before designing one, we needed to define exactly what it would be expected to detect.
Chapter 1 established that fluent generation can continue after evidential support has weakened or disappeared.
Chapter 2 showed that hallucination covers several different failure relationships.
Chapter 3 then separated the objects a reliability system must not collapse:
truth ≠ evidence ≠ support ≠ attribution ≠ provenance ≠ verification ≠ policy acceptance Now we can finally ask the engineering question:
What can software actually measure?
Suppose we have a resolved claim:
Company A acquired Company B in 2024.
A reliability system might inspect:
word overlap embedding similarity entailment probability retrieval relevance source metadata agreement across independent sources model confidence variation across repeated generations semantic entropy another model's judgment hidden-state activations counterfactual response sensitivity an executable database query execution traces geometric distance from an evidence representation All of those can be useful.
They are not measurements of the same thing.
That gives us the central rule of this chapter:
A hallucination measurement is a sensor, not a verdict.
Every sensor observes some property of a candidate through some observation channel.
Every sensor throws information away.
The job is therefore not to find the metric with the most impressive name.
It is to specify what the metric can observe, what latent property we hope it approximates, and where those two can come apart.
That separation is the beginning of measurement.
1. Every detector lives across a proxy gap What we care about is usually latent.
We want to know things such as:
Is this claim true? Is it supported by the evidence? Does the model know enough to answer? Did the claimed tool action really happen? Does the answer respond to the decisive facts of the problem? What software actually receives are observable signals.
For example:
Latent target Observable proxy Claim support NLI score, LLM support judgment External factuality Retrieval + evidence verification Epistemic uncertainty Semantic entropy, calibrated self-evaluation Semantic containment Projection residual against evidence representations Runtime fidelity Execution trace / authoritative state Context sensitivity Response change under controlled perturbation The dangerous move is:
proxy ↓ correlates with target on one evaluation ↓ proxy gets renamed as target That is how we end up with claims such as:
high similarity = grounded high confidence = true high consistency = correct high retrieval score = evidence low semantic entropy = factual None of those implications holds in general.
We will call the distance between the property we care about and the signal we can observe the proxy gap.
A detector is useful when its proxy tracks the target sufficiently well under the conditions we care about.
A detector fails when the target and proxy decouple.
This gives us a scientific way to think about hallucination measurement:
Define the proxy. State why it should track the target. Then construct cases designed to make them diverge.
That is the method we will use for the rest of the book.
2. Measurement begins with target, reference, and granularity A score means nothing until we specify the measurement contract that produced it.
For sensor \(i\), write:
$$ m_i(c,r_i,g_i) \rightarrow s_i $$where:
c = candidate claim or object under evaluation r_i = observation/reference available to sensor i g_i = measurement granularity s_i = observed score or state The reference might be:
no external reference the model's token distribution other samples from the same model a supplied document a retrieved corpus a provenance graph a knowledge base a runtime trace the open web The granularity might be:
token claim claim–passage pair claim triplet sentence document response pair source set trajectory This dimension matters.
A sentence-level entailment model can contain useful signal and still fail if it is asked to judge an entire document as one undifferentiated unit.
A claim detector can be excellent while the upstream claim extractor silently merges two propositions or drops a numerical qualifier.
So the unit is not a formatting detail.
Granularity is part of the measurement.
A resolved claim in this chapter means that the system has already tried to identify the proposition being judged: entities, relation, polarity, scope, time, quantity, modality, and relevant qualifiers. That resolution can itself be wrong, which means extraction and normalization are part of the measurement pipeline rather than infallible preprocessing.
3. A score is not a decision Suppose a detector returns:
0.82 What does that mean?
Without a contract, almost nothing.
It could mean:
82% lexical overlap cosine similarity of 0.82 82% entailment probability 82% estimated correctness probability 82% agreement across samples 82% of extracted atomic claims supported Even when the metric is known, the number is still not an acceptance decision.
The architecture should remain:
graph LR C[candidate + observation] --> M[measurement] M --> S[score / typed state] S --> CAL[calibration] CAL --> POL[policy + intended action] POL --> D[decision] Calibration turns detector output into policy input by making scores comparable to an explicit decision threshold.
Chapter 6 will handle calibration formally. For now, consider the difference between these two designs:
# Unjustified if score > 0.5: reject() and:
threshold = calibrated_threshold( detector=detector, domain=current_domain, target_false_accept_rate=0.05, ) decision = policy.route( measurement=score, threshold=threshold, intended_action="publish", ) The second version is still simplified, but at least it exposes the missing questions.
A continuous score has to earn its operational meaning through evaluation.
And two additional ideas will matter later:
discrimination How well does the score rank risky and safe examples? calibration Does a stated probability correspond to observed frequencies? A detector can be good at one and poor at the other.
We will measure both later.
4. Start from the Claim Verification Object, not from a favorite detector Chapter 3 built an evidence-bearing candidate rather than passing raw prose through the system.
That object already tells us which measurements are needed.
Conceptually:
CVO field / unresolved question Natural sensor class No evidence attributed to claim retrieval / attribution Claim–evidence relation unknown NLI, evidence-aware judge, structured relation check Evidence may be stale or circular provenance / metadata / source graph Sources disagree cross-source coherence / conflict measurement Verification path exists but was not executed runtime assertion / tool trace Claim can be compiled into a query SQL / programmatic / symbolic verification Model may not know enough confidence, self-evaluation, semantic entropy Answer ignores decisive context counterfactual sensitivity Claim may extend beyond evidence representation geometric containment This is a much better starting question than:
Which hallucination library should I install?
The CVO identifies an unresolved field.
The unresolved field determines the target property.
The target property determines which class of sensor can possibly help.
That gives us a practical selection rule:
Choose the cheapest sensor that observes the missing property with enough reliability for the intended action.
Not every claim needs every detector.
A runtime fact that can be checked with an exact trace assertion should not be sent through five probabilistic judges first.
A historical claim that exists only in external records cannot be solved by inspecting a local execution trace.
The reference chooses the sensor.
5. Primitive measurements compose into verification pipelines Several methods are often presented as distinct hallucination detectors even though they are compositions of a smaller set of operations.
Useful primitives include:
D = decompose response into claims R = retrieve candidate evidence A = attribute claim to evidence S = estimate claim–evidence relation P = inspect provenance / freshness / independence U = estimate model uncertainty C = compare repeated or counterfactual outputs V = verify against authoritative state Then familiar systems become compositions.
For example:
embedding retrieval = R NLI support checker = S_nli FActScore-style factuality = D → R → S → aggregate LLM judge with supplied evidence = S_llm LLM judge with web access = R → A → S_llm runtime validation = V This matters because composite detectors fail compositionally.
A reported factuality score might be wrong because:
claim extraction failed or retrieval missed the evidence or attribution linked the wrong passage or support classification failed or aggregation hid one critical error We will call this measurement-chain error.
A false alarm from a composite detector does not automatically prove the generator behaved correctly.
It may prove that the verifier failed.
Likewise, a passing score can hide a broken upstream stage.
A production reliability system therefore needs stage-level telemetry rather than only a final factuality number.
6. Authoritative checks: use software when software already knows Before reaching for semantic measurements, check whether the surrounding system already possesses the authoritative state.
Suppose the model says:
I ran the test suite and all 214 tests passed.
If the runtime has a trustworthy execution trace, ask:
Was the test tool called? Which command executed? What exit code returned? How many tests were reported? Likewise:
model says file exists → filesystem state model says database returned 143 rows → query-result object model says payment completed → transaction state model says attachment was inspected → input registry + trace These measurements can be exact because the relevant reference is explicit system state.
The priority rule is:
When the system already possesses authoritative state, compare against that state before reaching for probabilistic semantic detection.
There is one assumption underneath the rule:
The instrumentation itself must be complete, correctly associated with the run, current, and trustworthy.
A missing trace event proves little if the tracing system drops events.
Deterministic verification is conditional on reliable instrumentation.
That is still an enormous advantage over semantic guesswork.
7. Programmatic verification: compile claims into checks when possible Some claims are not runtime observations but can still be reduced to executable checks.
For example:
Claim: Revenue in 2024 was $5.2 billion. may become:
SELECT revenue_usd FROM annual_financials WHERE company_id = :company AND fiscal_year = 2024; A mathematical claim may become a Python calculation.
A code claim may become a unit test.
A repository claim may become a file or AST query.
A structured event claim may become a database lookup.
Conceptually:
graph LR NLC[natural-language claim] --> CN[claim normalization] CN --> EQ[executable query / assertion] EQ --> AR[authoritative result] AR --> COMP[comparison] This family sits between free-form semantic checking and direct runtime assertions.
The main risk moves upstream:
Did we compile the natural-language claim into the right program?
If the query misrepresents the claim, exact execution only gives us an exact answer to the wrong question.
Programmatic verification is powerful precisely because it makes that translation inspectable.
8. Proximity signals: cheap, useful, and non-directional The simplest reference-relative measurements ask whether claim and evidence resemble one another.
Lexical overlap Token overlap, n-gram overlap, edit distance, and ROUGE-style metrics are:
cheap fast deterministic interpretable But consider:
Evidence: The study did not find a statistically significant reduction in mortality. Claim: The study found a statistically significant reduction in mortality. Almost every important token is shared.
The proposition has flipped.
Or:
Evidence: Company A acquired Company B. Claim: Company B acquired Company A. As a bag of words, the material is effectively identical.
The role assignment is reversed.
Lexical overlap therefore measures:
How much surface material is shared?
It does not reliably measure polarity, role, causality, temporal order, or evidential strength.
Embedding similarity Embeddings replace surface overlap with semantic proximity.
Let:
$$ \mathbf{c}=\operatorname{Embed}(c) $$and:
$$ \mathbf{e}=\operatorname{Embed}(e) $$A common signal is:
$$ \operatorname{cos}(\mathbf{c},\mathbf{e}) \frac{\mathbf{c}\cdot\mathbf{e}} {|\mathbf{c}||\mathbf{e}|} $$
This recognizes paraphrase far better than lexical overlap.
But semantic proximity is still not directional support.
A reversed acquisition, a negated finding, or an inflated causal statement can remain very close to the source in embedding space.
The proxy gap is now:
semantic proximity ≠ evidential licensing Pairwise cosine also compares one claim vector with one evidence vector.
Real evidence is usually a set:
$$ E=[\mathbf{e}_1,\mathbf{e}_2,\ldots,\mathbf{e}_k] $$A natural next question is whether the claim can be represented by the region or subspace defined by the entire evidence set rather than by one nearest passage.
That geometric move motivates Chapter 5.
It does not eliminate the relation-inversion problem.
A wrong proposition can remain semantically in-span.
9. NLI and entailment: make the direction explicit Natural-language inference gives us a more suitable abstraction for directional claim–evidence comparison.
Given:
premise = evidence hypothesis = claim an NLI model may output:
ENTAILMENT CONTRADICTION NEUTRAL Now:
$$ \operatorname{NLI}(e,c) \neq \operatorname{NLI}(c,e) $$A good model should distinguish:
Evidence: The treatment did not reduce mortality. Claim: The treatment reduced mortality. as contradiction rather than high similarity.
SummaC demonstrated another important point: earlier NLI-based factual-consistency systems had suffered from granularity mismatch. Sentence-level NLI models were being applied too coarsely to document-level problems. Segmenting and aggregating pairwise relations recovered useful signal.[1]
This means:
The quality of claim extraction and evidence segmentation upper-bounds the quality of downstream NLI measurement.
NLI is still a trained classifier.
It can inherit annotation artifacts, lexical shortcuts, negation heuristics, domain shift, and prior biases from its training data.
It may also struggle with:
multi-hop evidence specialized domains numeric ranges temporal ordering long contexts scope and modality implicit assumptions Most importantly, evidential support is broader than textual entailment.
Evidence can increase the rational plausibility of a claim without logically entailing it.
Three randomized trials may support:
The intervention probably reduces hospitalization.
without making that statement a theorem logically entailed by one sentence.
So NLI is best understood as an operational sensor for a particular class of directional relation.
It is not a complete theory of evidence.
10. Retrieval is evidence discovery, not evidence judgment A retrieval score answers a search question:
Which evidence candidates should we inspect?
It does not answer:
What relationship do those candidates have to the claim?
A passage returned with relevance score 0.91 may:
support the claim refute the claim discuss the topic without deciding it be stale be circular come from a weak source This is why retrieval should remain a primitive rather than being renamed factuality.
A stronger pipeline looks like:
graph LR C[claim] --> RET[retrieve] RET --> AT[attribute] AT --> AS[assess support / refutation] AS --> PR[inspect provenance / freshness] PR --> AG[aggregate evidence state] Retrieval only becomes useful for factuality after the retrieved material is linked to the claim, checked for support, and filtered by provenance before aggregation.
Retrieval-backed factuality FActScore is a clear example of composition. It decomposes long-form output into atomic factual units and measures the fraction supported by a knowledge source, with an automated retrieval-plus-model estimator for large-scale use.[2]
Here, atomic means approximately one independently checkable proposition rather than an entire paragraph containing several relations.
Conceptually:
graph LR LA[long answer] --> AC[atomic claims] AC --> RET2[retrieve per claim] RET2 --> JUD[judge support] JUD --> AGG[aggregate factual precision] The architecture is powerful.
Its failure surface is equally important:
D: claim extraction error R: retrieval miss A: wrong attribution S: support-classification error P: stale or circular source G: aggregation hides a critical minority error A low score may mean the generator hallucinated.
It may also mean the measurement pipeline failed to find or recognize real support.
That ambiguity must remain visible during evaluation.
11. Source and provenance sensors: measure the evidence path itself Chapter 3 separated support from source fitness.
That distinction needs its own measurements.
Many useful source checks are ordinary metadata operations:
publication date version / revision source type retraction status author / organization chain of custody independence from claim origin content hash publication order These can answer questions such as:
Is this source stale for a current-state claim? Was this paper retracted? Are three apparent sources actually copies of one origin? Did the source exist before the claim it is supposedly verifying? Does policy require a primary source here? Circularity is especially important:
AI generates claim X ↓ website copies X ↓ search retrieves website ↓ AI cites website as evidence for X A semantic support detector may see exact textual agreement.
A provenance graph may reveal that the evidence path has no independent root.
Again, different sensor, different property.
Cross-source conflict Evidence can also disagree with itself.
For claim \(c\), a system can preserve counts or weights such as:
sources supporting c sources refuting c sources neutral / insufficient sources rejected for provenance Mixed high-quality evidence is not the same state as no evidence.
The correct action may be:
SURFACE_CONFLICT rather than:
ACCEPT or:
HALLUCINATION Source-set coherence is therefore another measurable property.
12. Self-consistency and semantic entropy: from string stability to meaning stability When external evidence is absent or expensive, we can use the model’s own stochastic behavior as an observation channel.
Self-consistency SelfCheckGPT samples multiple responses from a black-box model and looks for divergence or contradiction. Its core intuition is that a model with stable knowledge may produce mutually compatible samples, while confabulation may produce inconsistent ones.[3]
Conceptually:
graph LR P[prompt] --> A1[answer_1] P --> A2[answer_2] P --> A3[answer_3] P --> A4[answer_4] A1 --> CM[consistency measurement] A2 --> CM A3 --> CM A4 --> CM Answer consistency is a useful signal only after it is separated into surface stability, semantic stability, and calibration rather than treated as truth by repetition.
The strongest interpretation is:
How stable is the model’s answer under resampling?
Not:
Is the answer true?
For example:
sample 1 → Company B acquired Company A sample 2 → Company B acquired Company A sample 3 → Company B acquired Company A sample 4 → Company B acquired Company A If the real event ran in the opposite direction:
self-consistency = high factuality = low The temperature trap Repeated-sample methods need meaningful stochastic variation.
If decoding is fully deterministic, repeated generations collapse to the same path and the method loses the variation it was designed to inspect.
SelfCheckGPT is explicitly sampling-based.[3] Farquhar and colleagues likewise estimate semantic entropy from multiple sampled sequences; their reported implementation samples at temperature 1 for the uncertainty estimate.[4]
This does not mean there is one universally correct temperature.
It means the sampling scheme is part of the measurement contract.
Semantic entropy String variation can overstate uncertainty:
Paris It is Paris. France's capital, Paris. Kuhn and colleagues, and then Farquhar and colleagues, address this by clustering sampled responses into semantic equivalence classes — using bidirectional entailment to decide equivalence — and estimating entropy over those meaning clusters rather than over exact strings.[4][11]
At a conceptual level:
$$ H_{\text{semantic}} =-\sum_j p(z_j)\log p(z_j) $$where \(z_j\) denotes a semantic answer class rather than a literal string.
The result is a better proxy for semantic uncertainty.
The boundary is explicit in the original work: semantic entropy targets confabulations, not every systematically wrong answer.[4]
So:
high semantic entropy → unstable semantic answer / confabulation risk low semantic entropy → stable semantic answer ↛ truth A stable misconception can remain low entropy.
13. Confidence requires three different questions The word confidence hides several distinct measurements.
Token or sequence likelihood The language model directly provides something like:
$$ P(x_t \mid x_{