How to Evaluate a Hallucination Detector

Chapter 5 gave us something concrete enough to evaluate.

Hallucination Energy has:

a target property
an observable proxy
a mathematical definition
an implementation
continuous outputs
natural-task signal
adversarial failures

That sounds like progress.

It is also the point where it becomes dangerously easy to fool ourselves.

Suppose a detector reports:

AUC = 0.75

Is that good?

Perhaps.

But we still do not know:

which class was called positive
how hard the negatives were
whether train and test shared source documents
whether they shared entities or semantic neighborhoods
what prevalence the deployment actually has
whether the labels were trustworthy
how wide the confidence interval was
what happens at 1% false acceptance
how uncertain that 1% estimate is
whether the threshold transfers to another domain
whether the score is a probability at all
whether the verifier itself failed upstream
whether the generator changed
whether decoding parameters changed
whether the model collapses on structural adversaries

A single number cannot answer those questions.

So this chapter begins with a stronger rule:

A detector is not evaluated when we have computed a metric. It is evaluated when we understand how its errors behave under the conditions in which policy will use it.

And one more distinction will govern everything that follows:

    graph LR
    MQ[measurement quality] --> DQ[decision quality]
    DQ --> DU[deployment utility]
  

Good measurement is only the first dependency: the system still needs a decision rule that fits the operational costs of deployment.

1. Evaluation needs a contract too

Chapter 4 gave every detector a measurement contract.

Evaluation needs the same discipline.

But we should preserve the architecture established in Chapters 2 and 3.

A benchmark label such as:

SUPPORTED

is a measurement-relative class.

It is not automatically the same thing as:

SAFE

A supported claim may come from an inadmissible source.

An unsupported proposition may still be acceptable if clearly routed as a hypothesis rather than published as fact.

So the contract should speak measurement language first and policy language separately.

A stronger evaluation contract looks like:

evaluation_contract = {
    "target_property": "embedding_subspace_containment",
    "unit": "claim_evidence_pair",
    "reference_classes": {
        "positive": "unsupported",
        "negative": "supported",
    },
    "score_direction": "higher_means_more_risky",
    "reference_regime": "gold_or_resolved_evidence",
    "label_definition": None,
    "split_strategy": {
        "primary": "source_document_grouped",
        "entity_overlap_max": None,
        "semantic_cluster_holdout": False,
    },
    "prevalence_regimes": [0.50, 0.05, 0.01],
    "generator": {
        "model": None,
        "model_version": None,
        "temperature": None,
        "top_p": None,
    },
    "primary_metrics": ["roc_auc", "average_precision"],
    "operating_points": [
        {"max_false_accept_rate": 0.01},
        {"max_false_accept_rate": 0.05},
    ],
    "risk_control": {
        "method": None,
        "alpha": None,
        "assumptions": None,
    },
    "uncertainty": "group_bootstrap_95_ci",
    "adversarial_budget_sweep": [1, 4, 16, 64, 256],
    "parametric_leakage_probe": "counterfactual_entity_swap",
    "transfer_tests": [
        "source",
        "domain",
        "embedder",
        "generator",
        "decoding",
        "negative_difficulty",
        "failure_mechanism",
    ],
    "policy_mapping": None,
}

The None values matter.

They prevent us from pretending a control exists merely because we know it ought to exist.

At minimum, an evaluation must declare:

1. What property is the detector supposed to rank or classify?
2. What is the evaluation unit?
3. What are the benchmark classes?
4. Which score direction means greater risk?
5. How were labels established?
6. How are model selection, calibration, and final test data separated?
7. What dependency is the split designed to prevent from leaking?
8. How difficult are the negatives?
9. Which operating points matter to policy?
10. How uncertain are those operating points?
11. Which forms of transfer must the detector survive?
12. How does policy map measurements into actions?

Without those answers, an AUC is metadata without a contract.

2. Separate the detector confusion matrix from the policy confusion matrix

This distinction is easy to miss and fundamental to this book.

The detector answers a measurement question.

For example:

Does this claim appear supported by this evidence?

Its confusion matrix is therefore measurement-relative:

Policy asks a different question:

What should the system do with this candidate now?

Policy may incorporate:

support status
source reliability
provenance
intended action
risk level
human review availability

Its outcomes are therefore different:

A false acceptance is therefore fundamentally a gate or policy error.

The detector may have contributed to it.

But the two concepts are not identical.

That preserves Chapter 4’s rule:

The sensor is not the verdict.

For the simple containment experiments in Chapters 5 and 6 we often use a deliberately narrow policy mapping:

supported     → eligible for acceptance
unsupported   → review or reject

That is useful for evaluating a gate.

It should never be mistaken for a universal statement that supported means safe.

3. Fix one ROC orientation and map it to operational terms

Statistical notation becomes confusing quickly unless we choose one orientation and keep it fixed.

For this book’s detector analysis, define:

positive class = unsupported
negative class = supported

higher score = more risky

score > τ  → detector flags / rejects
score ≤ τ  → detector passes / accepts

Under that convention:

So:

This mapping matters because FAR is not the same quantity as ROC FPR here.

Operationally:

lower FAR
    usually requires
stricter threshold
    which usually causes
higher FRR / lower SAR

A detector that rejects everything has excellent FAR and terrible usefulness.

Reliability is not maximal rejection.

4. Ranking quality and decision quality are different questions

A continuous detector produces a score.

For Hallucination Energy:

low energy  → more contained
high energy → less contained

Before choosing a threshold, we can ask whether the score ranks unsupported examples above supported ones.

That is what ROC-AUC summarizes.

For a risk score \(s\) where higher means riskier:

Then:

AUC = 0.5
→ random ordering

AUC > 0.5
→ useful ordering signal

AUC < 0.5
→ ordering is reversed on that evaluation distribution

A globally reversed score can be reoriented by replacing \(s\) with \(-s\).

That is not the interesting failure.

The serious failure is:

the relationship between score and risk changes direction across domains or failure mechanisms.

That is what CaseHOLD warns us about.

A score that behaves one way on factual containment and another way on relation-heavy legal examples has suffered a representation or mechanism-transfer failure.

A threshold cannot repair that instability.

But even a strong AUC still does not tell us which threshold is acceptable.

AUC integrates ranking behavior across many possible thresholds.

Production policy normally cares about a narrow region such as:

FAR ≤ 1%

or:

FAR ≤ 0.1%

A detector can have respectable global AUC and perform poorly in that region.

AUC is not a false-acceptance budget.

5. The Wikipedia result: good separation, weak strict-policy coverage

Recall the hard-mined Wikipedia / FEVEROUS-style result from Chapter 5:

supported mean energy      = 0.3714
hard-negative mean energy  = 0.6950
mean gap                   = +0.3236
standardized separation    ≈ +1.92

These are semantic-neighbour hard negatives — hard_mined_v2, not the energy-aware adversary — recorded in the Certum artifact adversarial_consolidated.csv. Read them with the same caution Chapter 5 attaches to them.

Those distributions show real signal.

But the reported held-out operating point under that calibration configuration was:

threshold                  = 0.1468
supported acceptance rate  = 0.067
false acceptance rate      = 0.006

So:

~0.6% of unsupported examples were accepted

but only

~6.7% of supported examples were accepted

There is no contradiction.

The means can be separated while the distributions still overlap badly in the low-energy tail where a strict acceptance policy operates.

Distributional separation is not an operating point.

A policy report should therefore make the threshold path explicit:

    graph LR
    CD[calibration domain] --> CRT[calibration risk target]
    CRT --> ST[selected threshold]
    ST --> HE[held-out evaluation domain]
    HE --> FAR[observed FAR + uncertainty]
    FAR --> SAR[observed SAR + uncertainty]
  

A metric table becomes useful only after each reported cell is connected to the operational error rate it implies.

A useful table is:

The threshold belongs to calibration.

The reported behavior belongs to held-out evaluation.

Mixing those stages makes the operating point look more certain than it is.

6. Prevalence changes what alerts mean

ROC ranking is relatively insensitive to class prevalence.

Production alert quality is not.

This is the base-rate problem, and it is one of the easiest ways to misread a detector.

Let:

1 - FAR = unsupported detection rate
FRR     = supported false-alarm rate

The precision of an unsafe alert is therefore:

The denominator uses FRR, not FAR, because false alerts come from supported examples that the detector wrongly flags.

Suppose:

unsupported prevalence = 1%
unsafe detection rate  = 80%
FRR                     = 5%

In 10,000 candidates:

unsupported examples   = 100
caught unsupported      = 80
supported examples      = 9,900
false alerts            = 495

Alert precision is:

This is the base-rate fallacy in operational form:

A good-looking balanced-benchmark score does not imply a useful production alert stream.

For average precision, the naive random baseline is approximately the positive prevalence.

So with:

Always report prevalence with precision-recall metrics.

7. Threshold calibration is not probability calibration

The word calibration hides two different tasks.

Operating-point calibration

Suppose Hallucination Energy returns:

0.42

That is not intended to mean:

42% probability of hallucination

It is a geometric residual.

We can still choose a threshold from calibration data.

For a containment gate that accepts when:

The current Certum calibrator uses percentile-style thresholding over hard-negative energies.

This is operating-point calibration.

It maps a non-probabilistic score to a decision boundary under an empirical error budget.

Probability calibration

A different situation occurs if a model outputs:

P(unsupported) = 0.8

Now the number claims probabilistic meaning.

Among examples assigned probability around 0.8, we would like roughly 80% to actually be unsupported.

Metrics such as:

Brier score
reliability diagrams
Expected Calibration Error (ECE)

become relevant here.[2]

But ECE is itself a measurement with choices:

binning scheme
number of bins
sample size

So:

ECE = 0.03

should not become another magical scalar.

Use reliability diagrams and proper scoring rules alongside it.

And temperature scaling or any other calibration mapping must be fit on held-out calibration data, not final test data.

The central rule remains:

Do not compute probability-calibration metrics for an arbitrary detector score unless that score is intended to represent a probability.

8. Error budgets need uncertainty, not just empirical percentiles

Suppose a calibration set gives:

observed FAR = 0.6%

That does not establish:

future FAR ≤ 0.6%

It gives an estimate from a finite sample.

For a production claim we also need:

number of unsupported examples
confidence interval or upper bound
resampling / statistical method
exchangeability assumptions

At low error budgets, sample size becomes a first-principles constraint.

You cannot credibly estimate a one-in-a-thousand failure rate with fifty unsafe examples.

A useful intuition is the classical rule of three: if we observe zero failures among \(n\) independent trials, a rough 95% upper confidence bound is:

0 false accepts out of 1,000 unsupported examples

is compatible with an upper bound of roughly:

0.3%

rather than zero risk.[8]

For nonzero counts, use an appropriate binomial interval or another justified uncertainty method rather than extending the shortcut blindly.

A stronger policy condition is therefore:

9. Conformal risk control can strengthen threshold selection—but only under its assumptions

Percentile calibration is useful.

We can go further when the loss and data regime satisfy stronger assumptions.

Conformal Risk Control extends conformal ideas to control the expected value of bounded monotone losses with finite-sample guarantees under exchangeability.[9]

For our simple acceptance threshold, first condition on the unsupported calibration class.

Let the unsupported calibration examples be:

The standard CRC theorem is written for a non-increasing parameterization, so we can equivalently reparameterize with:

Under the theorem’s exchangeability and monotonicity conditions, this controls the expected future false-acceptance loss for examples drawn from the same unsupported distribution at level \(\alpha\), up to the finite-sample conformal construction.[9]

That statement is deliberately narrower than:

future production FAR is guaranteed forever.

CRC does not remove:

domain shift
mechanism shift
label noise
broken exchangeability
adversarial adaptation
instrumentation errors

And if we calibrate on the whole deployment mixture rather than the unsupported class, the same binary loss controls the unconditional probability:

The same machinery has been taken further. Mohri and Hashimoto apply conformal prediction directly to language-model factuality: they treat correctness as an uncertainty-quantification problem and derive a back-off procedure that removes or generalizes claims until a high-probability factuality guarantee holds.[12] That is the conformal idea used not to pick a threshold but to edit the output — a bridge to the repair and deletion strategies of Chapter 13.

Conformal risk control is therefore a powerful extension to the evaluation toolbox—not a license to stop testing transfer.

10. Model selection, calibration, and final test must be separated

A threshold is a learned parameter.

A feature set is a learned choice.

A rank cap can become a learned choice.

So can:

embedding model
classifier family
hard-negative generator
hyperparameters
aggregation rule

The stronger experimental structure is:

    graph LR
    DEV[development / model-selection data] --> CHOOSE[choose detector, features, rank, attack design]
    CAL[calibration data] --> THRESH[choose threshold / probability mapping]
    FINAL[FINAL LOCKED TEST] --> REPORT[report performance once]
  

The final test remains meaningful only if threshold selection and probability mapping have already been locked away from it.

Or use nested cross-validation where appropriate.

The principle is simple:

A test set stops being a test set when its score changes what you try next.

Repeated experimentation creates selection pressure even when nobody calls it training.

The best score after 100 attempts is not equivalent to one prespecified test.

Record how many decisions the test result influenced.

11. Row-level splitting can leak the thing you meant to test

Suppose a summarization dataset contains two outputs for the same source document:

source document X
    ├── supported summary
    └── hallucinated summary

Now imagine:

supported summary from X   → training
hallucinated summary from X → test

The rows are different.

The underlying evidence geometry is not.

A detector or downstream classifier can benefit from document-specific structure that leaks across the split.

So there are increasingly strong split regimes:

    graph TD
    RR[random row split] --> SD[grouped by source document]
    SD --> ED[entity-disjoint split]
    ED --> SC[semantic-cluster holdout]
    SC --> HT[held-out time period]
    HT --> HD[held-out domain]
    HD --> HG[held-out generator family]
  

Each stronger split removes another route by which the detector can appear to generalize while actually reusing structure from training.

Entity-disjoint splitting

Construct an entity co-occurrence graph in which documents sharing named entities are connected.

Then partition at the connected-component or graph-cluster level so important entities do not leak across train and test.

This is much harsher than source grouping.

It tests whether the detector generalizes beyond familiar entity neighborhoods.

Semantic-cluster holdout

Cluster source/evidence representations and assign whole semantic clusters to different partitions.

This reduces local semantic-neighborhood overlap.

It does not magically make train and test subspaces orthogonal.

It simply creates a stronger semantic extrapolation test than random rows.

This distinction is directly relevant to our own work.

The current Certum summarization evaluator uses:

70/30 stratified row-level train/test split

and:

5-fold stratified row-level cross-validation

with stratification by the benchmark class label.

Those results — the AUC table recovered from Certum summarization run 20260216_225831, which Chapter 5 Section 16 lists in full — describe row-level generalization under that pipeline.

They are not yet evidence of source-, entity-, or semantic-cluster-independent generalization.

So those HaluEval numbers should be read with that qualification, in Chapter 5 and here alike.

The split key should represent the dependency you are trying to prevent from leaking.

A 30% test set is not automatically a strong test set.

12. Statistical uncertainty has more than one source

Suppose two detectors report:

Detector A: AUC = 0.731
Detector B: AUC = 0.739

Is B better?

We cannot know from point estimates alone.

The current Certum evaluator bootstraps held-out AUC with 1,000 resamples and reports the 2.5th and 97.5th percentiles as a 95% interval.

That captures one useful form of uncertainty.

But there are at least two different questions.

Conditional evaluation uncertainty

Fix the already-trained detector.

Resample held-out groups and ask:

How much would the metric change if the test sample changed?

If rows within a source are dependent, use group bootstrap rather than row bootstrap.

Full-pipeline uncertainty

Repeat:

training / fitting
calibration
threshold selection
evaluation

across resamples or seeds.

This asks:

How much does the entire development process vary?

The second is more expensive and often wider.

It is also closer to the uncertainty of the actual pipeline we intend to redeploy.

For comparing correlated ROC curves on identical held-out examples, DeLong’s classical nonparametric method is useful.[3]

But a row-level DeLong or bootstrap analysis still does not repair group leakage.

The dependency structure comes first.

13. Easy negatives can manufacture excellent detectors

Imagine a detector for whether a claim is supported by evidence.

Positive pair:

Claim:
Company A acquired Company B in 2024.

Evidence:
Company A completed the acquisition of Company B in March 2024.

Now construct a negative by pairing the claim with:

Evidence:
The Atlantic Ocean is the second-largest ocean on Earth.

Almost any semantic detector will separate the two.

That does not prove hallucination detection.

It proves topic matching.

Negative construction therefore defines benchmark difficulty.

A useful difficulty ladder is:

LEVEL 0
random unrelated evidence

LEVEL 1
dataset-native mismatched evidence from another example

LEVEL 2
same-domain or same-topic mismatch

LEVEL 3
nearest-neighbor semantic mismatch

LEVEL 4
metric-aware hard-mined mismatch

LEVEL 5
structural adversary:
role inversion / negation / temporal reversal / recombination

A detector should be evaluated across this ladder.

Why?

Because high performance on Level 0 can coexist with total collapse on Level 5.

Recent hallucination benchmarks make the same difficulty issue visible. MedHallu explicitly constructs easy, medium, and hard medical hallucinations and reports substantially weaker performance on the hard category; its analysis finds the hard hallucinations are semantically closer to ground truth.[4]

The chapter’s difficulty ladder should therefore be crossed with another axis:

Failure-mechanism slicing

A global AUC can hide a detector that succeeds mainly because the benchmark contains many easy novel-entity errors while failing almost completely on role binding.

That would be a measurement failure disguised as a benchmark success.

14. Hard-negative mining must publish its search budget

Chapter 5 used hard_mined_v2 to build hard negatives for Hallucination Energy.

The procedure is approximately:

    graph TD
    C[for each claim] --> CS[compute centroid similarity to available evidence-set pool]
    CS --> SH[shortlist top K = 16 semantically similar candidate evidence sets]
    SH --> EX[exclude the claim's own evidence set, and any set sharing its source page]
    EX --> SEL[select the most similar remaining mismatched evidence set]
  

The selection criterion is similarity, not energy. This is a semantic-neighbour hard negative — Level 3 on the difficulty ladder above. The metric-aware version, which computes energy for each shortlisted mismatch and picks the minimum, is a separate mode (hardest_energy_mined, Level 4); it is not the one Chapter 5 reports.

Either way, difficulty depends on:

So an adversarial benchmark should record:

candidate pool
shortlist size K
similarity filter
metric used for mining
detector version attacked
adversary version
random seed
attack-development data
attack-test data
whether labels were used during construction

The attack itself also needs a train/test discipline:

attack development set
    ↓
design mining strategy

freeze adversary
    ↓
held-out adversarial test

Otherwise every observed weakness changes the attack, and the final number becomes a development diagnostic rather than an unbiased test estimate.

The Adversarial Degradation Curve

Plot:

K = 1
K = 4
K = 16
K = 64
K = 256

against:

AUC
FAR at fixed policy
SAR at fixed FAR target
mean separation

We can summarize one part of this curve with a book-specific diagnostic:

This is not a universal standard metric.

It is a useful way to state:

How quickly does the detector degrade as the adversary is allowed to search harder?

The curve is more informative than one K=16 score.

15. Test whether the verifier is reading the evidence or remembering the world

Reference-relative detectors can accidentally succeed for the wrong reason.

An LLM judge may mark:

Company Alpha acquired BetaCorp in 2024.

as supported partly because it already knows the real-world fact from pretraining rather than because it actually inspected the supplied evidence.

A useful probe is counterfactual entity substitution.

Transform:

Claim:
Company Alpha acquired BetaCorp in 2024.

Evidence:
Company Alpha completed the acquisition of BetaCorp in March 2024.

into:

Claim:
Company Xylophis acquired Zorbatek in 2024.

Evidence:
Company Xylophis completed the acquisition of Zorbatek in March 2024.

The local entailment relation is unchanged.

Parametric world knowledge is largely removed.

If an evidence-aware verifier collapses on this counterfactual form, something other than local support checking may be contributing to its performance.

Interpret the result carefully.

Synthetic names can themselves be out-of-distribution for an embedder or judge.

So the probe does not uniquely identify parametric leakage.

It tells us that local-evidence invariance failed, which is exactly the condition worth investigating.

16. Benchmark labels are measurements too

An evaluation cannot establish detector quality more precisely than its labels and reference procedure justify.

Suppose a benchmark labels a claim:

UNSUPPORTED

but its evidence package simply omitted the relevant source.

Or suppose it labels:

SUPPORTED

because one passage shares the topic but does not license the proposition.

Now the benchmark itself contains a measurement error.

Every label should therefore be understood as:

a judgment produced under a reference definition and annotation procedure

not direct access to truth.

A good evaluation report should record:

label source
annotation instructions
annotator count
adjudication procedure
reference evidence
claim granularity
NEI / refutation distinction
known or estimated label noise
label version

A practical audit is to independently re-annotate a stratified sample and report disagreement by failure mechanism.

Do not invent a label-noise percentage merely because a benchmark is popular.

HalluLens uses dynamic test-set generation for some extrinsic tasks partly to mitigate leakage and improve robustness.[5]

And the broader 2025 study Evaluating Evaluation Metrics – The Mirage of Hallucination Detection tests multiple hallucination metric families across datasets, model families, and decoding methods and finds substantial problems with human alignment, narrowness, and generalization.[6]

The concern is not only theoretical. A 2026 RAG-focused study rebuilds a hallucination-detection benchmark, adds controlled noisy-label variants, and reports that label noise measurably hinders detection performance — the benchmark’s own annotation error sets a ceiling on any detector score computed against it.[13]

The benchmark is another instrument in the chain.

It needs an audit too.

17. Evaluate the whole measurement chain: verifier error attribution

Chapter 4 introduced measurement-chain error.

A retrieval-backed hallucination detector may perform:

D = decompose response into claims
R = retrieve evidence
A = attribute evidence
S = judge support
P = inspect provenance
G = aggregate

Suppose the final system marks a true, supportable claim as unsupported.

Where did the failure occur?

Possible answers include:

D failed to extract the right claim
R missed the relevant source
A selected the wrong passage
S misclassified the relation
P rejected a valid source
G over-weighted one local error

If we evaluate only the final binary output, all of these become:

false alarm

That is not enough for engineering.

A concrete trace

Suppose the candidate says:

Company A acquired Company B in 2024.

The world and the correct filing support it.

But the verifier does this:

    graph TD
    D[extract claim] -->|PASS| R[retrieve evidence]
    R -->|FAIL: retrieves wrong filing| A[attribute evidence]
    A -->|FAIL downstream| S[judge support]
    S -->|returns INSUFFICIENT| P[inspect provenance]
    P -->|PASS| G[aggregate]
    G -->|marks candidate unsupported| E2E[END-TO-END FAIL]
  

The generator did not hallucinate.

The verifier failed retrieval.

A system that calls this simply:

model hallucination

will optimize the wrong component.

Serial pipeline ceilings

For a strictly serial pipeline in which each stage must succeed and later stages cannot recover from an earlier failure, define:

claim extraction success = 0.85
retrieval success        = 0.80
support judgment success = 0.98

implies a maximum all-three-success probability of:

The multiplication is exact only under the conditional definition above and the strictly serial no-recovery architecture.

Real systems may have retries, alternative routes, or recovery logic.

The general lesson survives:

Downstream classifier quality is upper-bounded by the information that survives upstream.

So stage-level telemetry is mandatory.

18. Transfer has at least three different failure modes

A threshold is conditional on the distribution used to choose it.

But not all distribution shift behaves the same way.

Prevalence shift

ROC ranking may stay stable.

Precision, alert burden, and expected cost can change dramatically.

Score / calibration shift

The detector may still work after recalibration.

Mechanism / concept shift

The relationship between observable signal and failure changes.

This is the CaseHOLD-style structural problem.

The score can stop ordering the classes sensibly.

Recalibration cannot manufacture a missing relational representation.

This gives a useful hierarchy:

prevalence shift
→ deployment metrics change

score shift
→ threshold recalibration may help

mechanism shift
→ detector itself may fail

Transfer testing should therefore vary:

new random sample
new source documents
new entities
new topic
new time period
new domain
new embedding model
new generator checkpoint
new decoding parameters
new failure mechanism

Model updates deserve special attention.

A vendor checkpoint change can alter:

answer style
hallucination mechanism
length distribution
uncertainty behavior
retrieval usage

without changing your application domain at all.

Evaluation is therefore not a one-time release ritual.

It is a monitoring obligation whenever the generator or verifier changes.

19. Selective prediction: reliability includes the option not to decide

Binary evaluation assumes every candidate must be accepted or rejected immediately.

Production systems often have another option:

REVIEW

or:

ABSTAIN

Suppose a detector is confident at the extremes but unreliable near its boundary.

We can define two thresholds:

score ≤ τ_accept
    → ACCEPT

τ_accept < score < τ_review
    → REVIEW / VERIFY

score ≥ τ_review
    → REJECT / ABSTAIN

Now we need more precise coverage terms:

auto-accept coverage
    P(ACCEPT)

auto-decision coverage
    P(ACCEPT or REJECT)

review rate
    P(REVIEW)

abstention rate
    P(ABSTAIN)

Risk–coverage analysis from selective classification formalizes exactly this trade-off between answering more examples and accepting more error.[10][11]

This is the mathematical implementation of a systems-level ability to say:

We do not have enough confidence in this measurement to automate the decision.

But review capacity is not infinite.

Alert fatigue is part of the risk model

A review rate of:

2%

may be operationally healthy.

A review rate of:

40%

may overwhelm the human queue.

Once reviewers begin rubber-stamping alerts to clear backlog, the nominal human safety layer no longer behaves like the assumed safety layer.

So review rate is not merely a monetary cost.

It is a capacity constraint and a potential failure mechanism.

The production question becomes:

How much work can this detector safely automate without collapsing the escalation path?

20. Cost belongs in evaluation

Two detectors can have similar quality and radically different deployment consequences.

Consider:

Detector A
one embedding + small matrix projection

Detector B
8 stochastic generations + web retrieval + frontier LLM judge

Even if B improves AUC, the engineering decision depends on:

latency
API cost
GPU cost
throughput
failure dependencies
privacy exposure
tool availability
human-review capacity

So evaluation should report an operational vector:

P(FA)     = P(candidate unsupported AND system accepts)
P(FR)     = P(candidate policy-admissible AND system rejects)
P(review) = P(system routes candidate to review)

This formulation intentionally uses joint deployment event probabilities so the cost terms share the same per-candidate units.

The numbers differ by application.

A false acceptance in casual brainstorming may cost little.

A false acceptance in medication advice, a financial filing, or production deployment can dominate every other term.

If:

C_FA  >> C_FR

policy should usually move toward stricter acceptance and more review.

If review itself is expensive or overloaded, the optimum changes again.

There is no application-independent optimal threshold.

Thresholds belong to policy because error costs belong to applications.

21. Audit the evaluator itself

One of the healthiest things we can do is apply this chapter to our own current pipeline.

The current Certum summarization evaluator already does several good things:

separates feature families
uses held-out train/test evaluation
reports ROC-AUC
bootstraps AUC with 1,000 resamples
runs 5-fold cross-validation
produces ROC and precision-recall plots
runs ablations
compares logistic and nonlinear models
records configuration

But an honest audit also exposes current limitations:

row-level rather than source-grouped splitting
row-level rather than group bootstrap
no final locked source-grouped test yet
no published single-Hallucination-Energy baseline yet
no systematic rank sweep yet
no embedding-model transfer sweep yet
no fixed-FAR curve with confidence bounds yet
hard-negative mining reported primarily at K=16
no adversarial K-sweep yet
no failure-mechanism slice table yet
no entity-disjoint or semantic-cluster split yet
no counterfactual local-evidence invariance probe yet

The next three evaluation upgrades are therefore unusually clear:

Priority 1 — dependency-correct splits

Run source-document-grouped train/test and group bootstrap.

Then compare against the current row-level result.

Priority 2 — operating-point curves with uncertainty

For multiple FAR budgets report:

threshold
FAR
FAR upper confidence bound
n_unsupported
SAR
FRR
review rate

Priority 3 — structural and budgeted adversaries

Evaluate:

role inversion
negation
quantity change
temporal reversal
cross-evidence recombination

across increasing search budgets.

This is not an embarrassment.

It is the purpose of evaluation discipline.

The chapter is not asking:

Can we make the current result look finished?

It is asking:

What evidence would make the claim stronger?

22. A minimum evaluation suite

We can now define a practical minimum suite for a new hallucination detector.

A. Measurement sanity

• target property declared
• score direction verified
• benchmark class orientation fixed
• single-feature baseline reported
• simple similarity baselines reported
• score distributions by class shown

For Hallucination Energy this means at least:

single energy scalar
max cosine
mean cosine
centroid similarity / distance
geometry feature bundle

B. Ranking

• ROC-AUC
• precision-recall / average precision
• prevalence reported
• confidence interval
• correlated-detector comparison when relevant

C. Operating points

At multiple policy budgets report:

• threshold
• FAR
• FAR confidence interval / upper bound
• number of unsupported examples
• SAR
• FRR
• review / abstention rate

D. Selection discipline

• model-development data separate
• calibration data separate
• final test locked
• number of major model/feature choices documented

E. Split robustness

Evaluate at least:

• random row split
• grouped source split

and when relevant:

• entity-disjoint split
• semantic-cluster holdout
• temporal split
• domain-held-out split
• generator-held-out split

F. Difficulty ladder

• unrelated negatives
• dataset-native mismatches
• same-domain negatives
• semantic-neighbor negatives
• metric-aware hard negatives
• structural adversaries

G. Failure-mechanism slices

• novel entity
• unsupported event
• relation inversion
• negation
• quantity alteration
• temporal reversal
• cross-evidence recombination

H. Adversarial budget

• K recorded
• attack development separated from attack test
• adversarial degradation curve reported

I. Transfer

Change without silently reusing calibration:

• domain
• embedder
• generator checkpoint
• decoding parameters
• segmentation
• rank / hyperparameters
• failure mechanism

J. Pipeline diagnostics

For composite systems:

• extraction metrics
• retrieval recall
• attribution accuracy
• support-classifier metrics
• provenance accuracy
• aggregation errors
• policy errors
• failure stage recorded for false decisions

K. Operational cost

• latency
• cost
• throughput
• auto-accept coverage
• auto-decision coverage
• review burden
• alert-fatigue capacity

If a detector has not survived this suite, we may still have an interesting research signal.

We do not yet have a production reliability claim.

23. What a defensible evaluation artifact should contain

A final evaluation should be reconstructable from an artifact rather than from prose alone.

For example:

evaluation_report = {
    "detector": {
        "name": "hallucination_energy",
        "version": "...",
        "score_direction": "higher_is_riskier",
    },
    "measurement": {
        "embedder": "...",
        "embedder_version": "...",
        "segmentation": "sentence",
        "rank": 8,
    },
    "generator": {
        "model": None,
        "model_version": None,
        "temperature": None,
        "top_p": None,
        "seed": None,
    },
    "dataset": {
        "name": "...",
        "version": "...",
        "label_definition": "...",
        "n": 10000,
        "unsupported_prevalence": None,
    },
    "labeling": {
        "reference_definition": None,
        "annotation_procedure": None,
        "label_version": None,
        "estimated_label_noise": None,
    },
    "selection": {
        "development_split": None,
        "calibration_split": None,
        "final_test_locked": None,
        "hyperparameter_selection_data": None,
    },
    "split": {
        "strategy": "grouped_source_document",
        "group_key": "source_document_id",
        "entity_overlap": None,
        "seed": 42,
    },
    "ranking": {
        "roc_auc": None,
        "roc_auc_ci95": None,
        "average_precision": None,
        "ap_baseline_prevalence": None,
    },
    "operating_points": [
        {
            "target_far": 0.01,
            "threshold": None,
            "eval_far": None,
            "far_ci95": None,
            "far_upper95": None,
            "n_unsupported": None,
            "supported_acceptance": None,
            "false_rejection": None,
        }
    ],
    "risk_control": {
        "method": None,
        "alpha": None,
        "calibration_n_unsupported": None,
        "exchangeability_assumption": None,
    },
    "prevalence_stress": {
        "p_50": None,
        "p_05": None,
        "p_01": None,
    },
    "stress_tests": {
        "adversarial_budgets": [1, 4, 16, 64, 256],
        "adversarial_decay": None,
        "structural_adversaries": None,
        "counterfactual_entity_swap": None,
        "held_out_domain": None,
    },
    "slices": {
        "relation_inversion": None,
        "negation": None,
        "quantity": None,
        "temporal": None,
        "recombination": None,
    },
    "pipeline": {
        "extraction_recall": None,
        "retrieval_recall_at_k": None,
        "attribution_accuracy": None,
        "support_accuracy": None,
        "failure_stage_counts": None,
    },
    "uncertainty": {
        "resampling_unit": "source_document",
        "n_resamples": 1000,
        "training_seed_sweep": None,
    },
    "operations": {
        "latency_p95_ms": None,
        "cost_per_1000_claims": None,
        "review_rate": None,
        "review_capacity": None,
    },
}

The None values are intentional.

If we did not measure something, the artifact should say so.

Unknown is better than invented precision.

24. Where Hallucination Energy currently stands

The framework in this chapter should be applied directly to the running detector.

What can we currently claim?

Demonstrated

non-random signal in natural summarization geometry features
clear distributional separation in some factual hard-negative settings
repeatable deterministic score computation under fixed configuration
known structural collapse on CaseHOLD-style relation-heavy examples
semantic-neighbour hard-negative construction at K=16
row-level held-out and cross-validated classifier evaluation
bootstrap AUC uncertainty under row resampling

Not yet demonstrated strongly enough

source-document-independent HaluEval generalization
entity-disjoint transfer
single-energy-scalar performance isolated from geometry bundle
stable rank sensitivity
embedding-model transfer
fixed low-FAR operating curves with confidence bounds
conformal risk-controlled acceptance
energy-aware (metric-adaptive) hard-negative mining
adversarial budget degradation curve
failure-mechanism slice robustness
stable performance across generator checkpoints / decoding regimes

So the correct current conclusion is neither:

Hallucination Energy is production-ready

nor:

Hallucination Energy failed

It is:

Hallucination Energy is a promising, bounded containment sensor whose trustworthy operating envelope is still being mapped.

That is a scientifically stronger statement because it says exactly what evidence exists and exactly what is still owed.

25. The standard for claiming a good detector

“Good” is not AUC > 0.8, not “works on one benchmark”, not “beats cosine similarity”.

A detector is good, for its declared target, when it has survived the minimum suite in Section 22 and can be reconstructed from the artifact in Section 23 — and when the report says, explicitly, where not to trust it.

That gives us the central principle of this chapter:

The purpose of evaluation is not to produce a flattering score. It is to discover the conditions under which a measurement remains trustworthy.

Chapter 5 gave us a perfect example.

Hallucination Energy showed useful signal on natural summarization and factual retrieval settings.

Then hard legal and structural examples exposed a representational ceiling.

A weak evaluation would have stopped at the first AUC.

A strong evaluation asks what happens next.

That is where we are going.

Research roots

This chapter combines general statistical evaluation principles with recent evidence that hallucination metrics can be brittle across datasets, models, and difficulty regimes.

1. Jesse Davis and Mark Goadrich, “The Relationship Between Precision-Recall and ROC Curves,” ICML 2006, pp. 233–240. Formalizes the relationship between PR and ROC spaces and helps explain why precision-recall analysis is particularly informative under class imbalance. https://doi.org/10.1145/1143844.1143874

2. Chuan Guo, Geoff Pleiss, Yu Sun and Kilian Q. Weinberger, “On Calibration of Modern Neural Networks,” ICML 2017, PMLR 70:1321–1330. Distinguishes predictive accuracy from probability calibration and demonstrates post-hoc calibration methods including temperature scaling. https://proceedings.mlr.press/v70/guo17a.html

3. Elizabeth R. DeLong, David M. DeLong and Daniel L. Clarke-Pearson, “Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach,” Biometrics 44(3), 1988, pp. 837–845. Classical nonparametric method for comparing correlated ROC-AUC estimates. https://pubmed.ncbi.nlm.nih.gov/3203132/

4. Shrey Pandit et al., “MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models,” EMNLP 2025. Builds controlled medical hallucinations at multiple difficulty levels and reports weaker detection on hard hallucinations that are semantically closer to ground truth. https://aclanthology.org/2025.emnlp-main.143/

5. Yejin Bang et al., “HalluLens: LLM Hallucination Benchmark,” ACL 2025. Develops a taxonomy-driven benchmark and uses dynamic test-set generation for extrinsic tasks to reduce leakage and improve robustness. https://aclanthology.org/2025.acl-long.1176/

6. Atharva Kulkarni et al., “Evaluating Evaluation Metrics – The Mirage of Hallucination Detection,” Findings of EMNLP 2025. Evaluates hallucination metric families across datasets, model families, and decoding methods and finds significant problems with human alignment, narrowness, and generalization. https://aclanthology.org/2025.findings-emnlp.1035/

7. Ernan Hughes, Certum, open-source implementation of claim–evidence geometry, percentile calibration, hard-negative mining, evaluation utilities, policy gating, and diagnostic traces. https://github.com/ernanhughes/certum

8. Ernst Eypasch, Rolf Lefering, C. K. Kum and Hans Troidl, “Probability of adverse events that have not yet occurred: a statistical reminder,” BMJ 311, 1995. Gives the familiar approximate 3/n upper 95% bound when zero events are observed. https://www.bmj.com/content/311/7005/619

9. Anastasios N. Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei and Tal Schuster, “Conformal Risk Control,” 2022. Extends conformal prediction to finite-sample control of expected bounded monotone loss under exchangeability, with extensions for several risk settings. https://arxiv.org/abs/2208.02814

10. Yonatan Geifman and Ran El-Yaniv, “Selective Classification for Deep Neural Networks,” 2017. Develops selective classification with a reject option and explicit risk/coverage control. https://arxiv.org/abs/1705.08500

11. Yonatan Geifman and Ran El-Yaniv, “SelectiveNet: A Deep Neural Network with an Integrated Reject Option,” ICML 2019, PMLR 97:2151–2159. Optimizes prediction and rejection jointly and evaluates risk–coverage trade-offs. https://proceedings.mlr.press/v97/geifman19a.html

12. Christopher Mohri and Tatsunori Hashimoto, “Language Models with Conformal Factuality Guarantees,” ICML 2024. Casts output correctness as uncertainty quantification over entailment sets and derives a conformal back-off procedure that removes or generalizes claims until a high-probability factuality guarantee holds. https://proceedings.mlr.press/v235/mohri24a.html

13. Wenbo Chen, Veena Padmanabhan, Tootiya Giyahchi, Elaine Wong and Leman Akoglu, “Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insights,” ACL 2026. Rebuilds a RAG hallucination-detection benchmark and releases sample-dependent noisy-label variants, reporting that label noise hinders measured detection performance. https://aclanthology.org/2026.acl-long.680/

Next: Breaking the Detector

Evaluation tells us how a detector behaves on the tests we thought to run.

Adversarial testing asks a more aggressive question:

Can we deliberately construct examples that exploit exactly what the detector cannot see?

Chapter 6 has now given us an attack specification:

choose a failure mechanism
        ↓
choose an adversarial budget
        ↓
freeze the detector version
        ↓
search for the cheapest counterexample
        ↓
measure degradation
        ↓
attribute the failure to representation, retrieval, calibration, or policy

For Hallucination Energy, the likely attack surface is already visible:

new semantic material
    → often raises energy

but

role inversion
polarity reversal
quantity changes
temporal reversal
cross-evidence recombination
    → may remain in-span

Chapter 7 will turn those observations into a systematic attack methodology.

We will move from:

sample the world

to:

actively search for counterexamples

and ask the question every reliability system eventually has to face:

What happens when the test data is designed to defeat the sensor rather than merely sample the world?