# Breaking the Detector
# Book: Hallucination From First Principles
# Chapter URL: https://programmer.ie/books/hallucination-from-first-principles/07-chapter/
# Prose fingerprint (sha256, normalized): 8a29b90c64ace09f3ca339f9fdc13b75e82c80572e0411457c50fa170dc259b0
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

Chapter 6 changed the standard for evaluation.
A detector is not trustworthy because it has a good AUC.
It becomes trustworthy only after we understand how its errors behave under the conditions in which policy will use it.
That immediately suggests a more aggressive question.
Instead of waiting for those conditions to appear naturally, can we search for them deliberately?
Suppose a containment detector works well on ordinary unsupported claims.
Do not celebrate yet.
Ask:
What is the smallest truth-bearing change that the detector fails to notice? What wrong evidence looks safest to this exact detector? What valid evidence can be removed before the score should become suspicious? What happens when the attacker sees the score and searches against it? What configuration change moves the decision boundary without changing the underlying claim? Which upstream verifier stage can be corrupted most cheaply? Ordinary evaluation asks:
How does the detector behave on examples we have?
Adversarial evaluation asks:
Can we construct valid examples specifically designed to exploit what the detector does not observe?
Hallucination Energy will remain the running victim because we know its mathematics, implementation and initial failure modes.
But the method is broader.
Every hallucination detector has an observation channel.
Every observation channel preserves some distinctions and discards others.
The red-team job is to find a failure that lives primarily in the discarded information.
A good adversarial example does not merely make the detector wrong. It explains why the detector could not see the failure.
That turns red teaming into more than attack generation.
It becomes experimental falsification for measurement systems.
1. Start with a threat model, not merely an attack idea Chapter 4 gave us a measurement contract.
Chapter 6 gave us an evaluation contract.
Adversarial testing needs an attack contract that also functions as a threat model.
A useful version looks like this:
attack_contract = { "victim": { "detector": "hallucination_energy", "detector_version": None, "embedder_version": None, "rank": None, "segmentation": None, "threshold": None, "calibration_version": None, }, "objective": { "target_failure": "unsupported_claim", "attack_goal": "false_acceptance", "oracle": "claim_evidence_support_label", }, "attacker": { "knowledge": "score_access", "capabilities": ["edit_claim", "choose_evidence_set"], "query_budget": 16, "text_edit_budget": None, "evidence_addition_budget": None, }, "validity_constraints": [ "oracle_label_preserved_or_changed_as_declared", "reference_remains_fixed_unless_evidence_is_attack_surface", "example_remains_well_formed", ], "evaluation": { "victim_frozen_before_attack": True, "threshold_frozen_before_attack": True, "held_out_attack_set": True, }, } Three attacker properties should remain separate.
Knowledge What can the attacker observe?
decision only score algorithm and configuration embedding representation internal activations / gradients This gives a useful spectrum:
decision-based ↓ score-based ↓ algorithm-aware ↓ white-box internals OpenHalDet makes a related distinction when comparing black-box, gray-box and white-box hallucination detectors under different model-access regimes.[7]
Capability What may the attacker change?
claim text retrieved evidence source ordering evidence duplication segmentation rank threshold configuration upstream retrieval A user may control only the prompt or claim.
A malicious document may influence retrieval.
A developer controls rank and chunking.
A vendor update may change model behavior without anybody intentionally attacking the system.
Budget How much pressure may the attacker apply?
number of detector queries candidate substitutions text edits evidence additions retrieval attempts compute / wall-clock budget These distinctions tell us what kind of result we have found.
A counterexample can be:
a realistic security threat a robustness stress test or a scientific falsification of an over-broad detector claim Those are all valuable.
They are not the same claim about production risk.
2. Separate baseline errors from attack-induced failures A subtle mistake can make an adversary look stronger than it is.
Suppose an unsupported example already passes the detector:
threshold = 0.20 original score = 0.12 original decision = ACCEPT An attack changes the score to:
adversarial score = 0.11 The adversarial example is still a false acceptance.
But the attacker did not cause the decision failure.
The detector was already wrong.
So adversarial reporting should distinguish at least three quantities:
baseline failure attack-induced failure worst-case adversarial failure Let the decision rule be:
$$ D_\tau(x)= \begin{cases} \operatorname{ACCEPT}, & s(x)\le\tau\\ \operatorname{REJECT}, & s(x)>\tau \end{cases} $$for a risk score where higher means riskier.
False-acceptance attack The adversary searches for an unsupported adversarial example:
$$ \min_{x'\in\mathcal A(x)}s(x') $$subject to:
$$ y(x')=\operatorname{UNSUPPORTED}. $$The overall adversarial false-acceptance rate is:
$$ \operatorname{ASR}_{FA}^{all}(\tau) P(D_\tau(x_{adv})=\operatorname{ACCEPT} \mid y(x_{adv})=\operatorname{UNSUPPORTED}). $$
But the attack-induced success rate asks a sharper question:
$$ \operatorname{ASR}_{FA}^{induced}(\tau) P( D_\tau(x_{adv})=\operatorname{ACCEPT} \mid D_\tau(x)=\operatorname{REJECT}, ,y(x_{adv})=\operatorname{UNSUPPORTED} ). $$
Now the attacker receives credit only when it flips a previously correct decision.
False-rejection attack The symmetric attack preserves support while making the example look risky:
$$ \max_{x'\in\mathcal A(x)}s(x') $$subject to:
$$ y(x')=\operatorname{SUPPORTED}. $$Its induced success rate is:
$$ \operatorname{ASR}_{FR}^{induced}(\tau) P( D_\tau(x_{adv})=\operatorname{REJECT} \mid D_\tau(x)=\operatorname{ACCEPT}, ,y(x_{adv})=\operatorname{SUPPORTED} ). $$
False rejection matters because a reliability system can be defeated by becoming unusable as well as by becoming permissive:
supported claims ↓ artificially high risk ↓ review overload ↓ coverage collapse ↓ alert fatigue ↓ human rubber-stamping or system abandonment Therefore report both directions.
And freeze $\tau$ before the adversarial test.
If the threshold changes after seeing attacks, the experiment has become detector tuning.
The overall workflow for a clean adversarial test is:
graph TD A[one successful flip] --> B[attack-induced success] B --> C[report ASR] C --> D[freeze threshold before test] Attack success is meaningful only when the threshold is fixed before measurement; otherwise the adversarial test becomes another tuning loop.
3. Organize attacks around what the measurement consumes A long list of attack names is difficult to reason about.
A more useful structure mirrors the measurement itself.
For Hallucination Energy:
$$ H(c,E\mid\phi,g,r) $$where:
c = claim E = evidence set φ = embedding representation g = evidence construction / segmentation r = retained rank Policy later adds calibration and thresholding.
That gives us three broad attack families.
Family What changes? Examples Content attacks truth-bearing structure of the claim role swap, negation, quantity, time, causal direction, binding Context attacks evidence supplied to the detector wrong neighbor, support deletion, inflation, echo, duplication, distractors Configuration / pipeline attacks representation or system around the detector segmentation, rank, embedder, threshold reuse, retrieval, attribution, aggregation The difficulty ladder from Chapter 6 now becomes search pressure inside those families:
LEVEL 0 unrelated mismatch LEVEL 1 dataset-native mismatch LEVEL 2 same-domain mismatch LEVEL 3 semantic-neighbor mismatch LEVEL 4 detector-aware search LEVEL 5 structural adversary The important progression is not the label number.
It is that each step removes another shortcut.
Adversarial difficulty should progressively preserve what the detector sees while changing what the detector is supposed to decide.
4. Content attacks: preserve the semantic material, change the proposition The cleanest attack changes the truth-bearing structure while preserving almost everything a broad semantic representation is likely to preserve.
Relation inversion Evidence: Company A acquired Company B. Supported: Company A acquired Company B. Adversarial: Company B acquired Company A. The entities are identical.
The event vocabulary is identical.
The role assignment changed.
Polarity reversal Evidence: The study did not find a significant reduction in mortality. Adversarial: The study found a significant reduction in mortality. One small edit can reverse the proposition while leaving lexical overlap almost perfect.
Quantity mutation Evidence: Revenue increased by 12%. Adversarial: Revenue increased by 21%. Temporal inversion Evidence: Regulatory approval occurred after the trial ended. Adversarial: Regulatory approval occurred before the trial ended. Causal reversal Evidence: A reduction in demand caused prices to fall. Adversarial: Falling prices caused demand to decline. These transformations share one principle:
Change the truth-bearing structure while preserving the representation-bearing surface.
Binding attacks Relation inversion is one special case of a deeper problem.
Suppose the evidence says:
Alice founded Alpha. Bob founded Beta. Now assert:
Alice founded Beta. Every semantic ingredient is present:
Alice Bob Alpha Beta founded The problem is which pieces are bound together.
The evidence encodes:
$$ \operatorname{Founded}(Alice,Alpha) $$and:
$$ \operatorname{Founded}(Bob,Beta) $$while the adversarial claim asserts:
$$ \operatorname{Founded}(Alice,Beta). $$This generalizes into cross-evidence recombination:
entity from passage 1 + relation from passage 2 + quantity from passage 3 + wrong binding This is particularly dangerous in multi-document RAG.
More retrieval creates more legitimate semantic components.
That can enlarge the space of unsupported propositions that can be assembled from individually real pieces.
More relevant evidence can increase the space of plausible recombinations.
Measure attack minimality A structural counterexample becomes more diagnostic when the edit is small.
Record:
tokens changed characters changed entities preserved relation vocabulary preserved surface syntax preserved embedding similarity before / after Embedding similarity here is a diagnostic, not the oracle.
If a one-token negation flips truth while the detector score barely changes, that is stronger evidence of a missing representational distinction than a complete rewrite.
5. Context attacks: change the evidence while keeping the claim fixed A claim–evidence detector assumes that the evidence package is an appropriate reference.
Attack that assumption directly.
Nearest-wrong evidence Two Certum procedures build a wrong evidence set for a claim.
hard_mined_v2 is the semantic-neighbour version. For each claim it approximately performs:
graph TD C[claim] --> R[rank evidence sets by centroid similarity] R --> SH[shortlist top K = 16] SH --> EX[remove the correct evidence and any set sharing its source page] EX --> SEL[select the most similar remaining mismatch] This is the version Chapters 5 and 6 report. It picks a plausible wrong reference, but it does not consult the detector.
hardest_energy_mined is the detector-aware version. It computes Hallucination Energy for each shortlisted mismatch and selects the minimum:
$$ E_{adv}(K) \arg\min_{E’\in\mathcal N_K(c)}H(c,E’). $$
That chooses the semantically plausible negative that this exact detector finds safest. It is the stronger attack, and it has not yet been run at the scale of the three-domain table. The energy-aware search sits at the top of the difficulty ladder from Chapter 6; the historical hard_mined_v2 result sits one rung lower, at the semantic-neighbour level.
Evidence inflation: permissiveness attack Add evidence directions that help explain an unsupported claim.
For the full span:
$$ V_E=\operatorname{span}(E) $$and:
$$ V_E\subseteq V_{E\cup D}. $$Therefore the orthogonal distance to the enlarged full span cannot increase:
$$ \left\|(I-P_{V_{E\cup D}})c\right\|_2 \le \left\|(I-P_{V_E})c\right\|_2. $$So full-span containment is monotone under evidence addition.
An adversary can search for $D$ such that:
$$ \min_{D:|D|\le b}H_{full}(c_{bad},E\cup D). $$This is a permissiveness attack: make a bad claim increasingly representable.
Evidence rotation: false-rejection attack Truncated SVD is different.
When only the top $r$ directions are retained, added passages can change singular values and rotate which directions survive.
Now a supported claim can become less contained because the fixed-capacity basis has been redirected toward distractors.
That is a separate attack:
$$ \max_{D:|D|\le b}H_r(c_{good},E\cup D). $$The distinction matters:
inflation attack → helps bad claim → exists even for full span if added evidence aligns with bad claim rotation attack → hurts good claim → depends on truncated representation / capacity allocation Support deletion Remove the one passage that carries the decisive relationship while leaving surrounding topical material intact.
before: 20 passages, one states A acquired B after: 19 passages still discuss A, B and the transaction but the decisive support is gone The detector should become more skeptical.
If the score barely changes, the measurement may be tracking topic containment rather than support-bearing content.
Evidence echo Append a passage that simply repeats the candidate claim.
Containment should improve.
That is not a containment bug.
The reference now literally contains the assertion.
The failure is elsewhere:
Was this passage independent evidence? Where did it come from? Did the generated claim create the source that is now being used to justify it? So evidence echo demonstrates:
Containment is not provenance.
Evidence duplication Repeat one evidence passage several times.
For a full span, duplication adds no new direction.
For truncated SVD, repeated rows change singular values and can alter which directions dominate the retained basis.
A large score movement under duplication reveals sensitivity to evidence weighting rather than new evidential content.
Distractor injection Add topically relevant but propositionally irrelevant passages.
Long-context RAG systems routinely produce this situation without an attacker.
A reliability system should not become more confident merely because the evidence package became longer and semantically broader.
6. Configuration and pipeline attacks: break the measurement around the metric Some of the most realistic failures do not mutate the claim at all.
They change the conditions under which the measurement is computed.
Segmentation Represent the same source as:
whole document sentences 50-token chunks 200-token chunks atomic propositions The evidence matrix changes.
Its spectrum changes.
Its effective rank changes.
The score can change even though the source facts do not.
Rank For a fixed nested SVD basis, Chapter 5 established:
$$ H_{r+1}(c,E)\le H_r(c,E). $$So sweep:
r = 1, 2, 4, 8, 16, 32, full and ask where supported and unsupported distributions converge.
Rank is not merely an implementation knob.
It controls how permissive the proxy can become.
Embedder transfer Run the same attack corpus through several embedding models.
A failure that disappears may be representation-specific.
A failure that persists across sentence encoders gives stronger evidence of a broader limitation.
Threshold reuse A calibrated threshold is also an attack surface when it is reused outside the distribution that justified it.
Conceptually:
calibrate on Wikipedia-like factual containment ↓ reuse unchanged in legal relational domain ↓ distribution / mechanism shift ↓ policy behavior no longer matches claimed risk budget This is often an accidental deployment attack rather than a malicious one.
The test is simple: freeze the threshold from domain A and report FAR/SAR on domain B rather than silently recalibrating.
Attack the verifier chain A complete verifier may contain:
D = decomposition R = retrieval A = attribution S = support judgment P = provenance G = aggregation Each stage has a natural attack:
Stage Attack Expected signature D — decomposition merge or omit a negated / qualified clause wrong atomic claim reaches verifier R — retrieval entity collision or semantic neighbor wrong evidence becomes reference A — attribution correct topic, wrong passage claim linked to non-supporting span S — support high-overlap contradiction or multi-hop relation support classifier mislabels relation P — provenance mirrors, derivative copies, circular sources repeated claim appears independent G — aggregation one critical false claim among many correct ones mean score hides high-severity failure PROBE makes a closely related process-level move by evaluating claim decomposition, evidence finding, evidence evaluation and hallucination localization separately rather than reducing the verifier to one judge call.[6]
A useful red-team record therefore asks two questions:
Which stage failed first? Which later failures were merely consequences of that corrupted state? For example:
retrieval returns wrong evidence ↓ attribution faithfully links to wrong evidence ↓ support classifier correctly judges that evidence ↓ aggregator produces wrong final decision Only the first stage may have been directly compromised.
The rest is error propagation.
That distinction prevents us from blaming the containment sensor for a retrieval failure—or praising downstream stages that never received the right evidence.
7. Search pressure should produce a curve, not one hard-negative score Once an attack searches over alternatives, the search budget becomes part of the result.
The historical result used a fixed shortlist of 16 and similarity-based selection. The Adversarial Degradation Curve from Chapter 6 §14 is the artifact that generalizes it: freeze the detector, embedder, rank, segmentation, threshold, candidate pool, and claim set, then sweep the budget.
For Hallucination Energy the sweep is concrete. For each:
K = 1, 4, 16, 64, 256 construct the shortlist and select the energy-aware mismatch:
$$ \mathcal N_K(c)=\operatorname{TopK}{E’\in P\setminus E{correct}} \operatorname{sim}(c,E’), \qquad E_{adv}(K) \arg\min_{E’\in\mathcal N_K(c)}H(c,E’). $$
Report ROC-AUC, the supported-vs-hard-negative gap, and ASR_FA_induced / FAR / SAR at the frozen threshold, at each budget, and plot them against $\log_2 K$. The curve is the artifact; its slope summary $\lambda_{adv}$ (Chapter 6 §14) is a convenience, not a replacement — the degradation need not be smooth, and a detector may have a knee or a mechanism-specific collapse.
graph LR SW[Freeze detector, embedder, rank, segmentation, threshold, candidate pool, claim set] SW --> SWEEP[K = 1, 4, 16, 64, 256] SWEEP --> REPORT[Report ROC-AUC, gap, ASR, FAR, SAR vs log2 K] REPORT --> CURVE[Adversarial Degradation Curve] The sweep treats attack budget as the independent variable, so the important result is how detector behavior degrades as harder negatives become available.
8. Metamorphic testing gives us expected relationships Many hallucination tests have an oracle problem because open-ended language has no single exact output.
Metamorphic testing helps by specifying a metamorphic relation: how a measurement should change, or should not change, after a controlled transformation.[1][2] The two families below are the standard invariance and directional relations from that literature.
Let $T$ be a transformation.
Define the score effect:
$$ \Delta s_T=s(T(x))-s(x). $$Invariance transformations The support label should remain unchanged.
We expect:
$$ |\Delta s_T| $$to remain small, subject to a tolerance learned from evaluation.
Examples:
meaning-preserving paraphrase consistent fictional entity renaming in claim and evidence evidence reordering formatting changes Sensitivity transformations The truth/support relation changes in a known direction.
For a higher-is-riskier detector we expect:
$$ \Delta s_T>0 $$when the mutation makes a previously supported candidate unsupported.
Examples:
subject/object swap negation quantity mutation temporal reversal remove sole support replace correct evidence with nearest wrong evidence This gives a compact test matrix:
graph TD subgraph Invariance PAR[Paraphrase] --> STAB[score approximately stable] REN[Consistent entity renaming] --> STAB ORD[Evidence reordering] --> STAB end subgraph Sensitivity SWAP[Subject/object swap] --> INC[risk should increase] NEG[Negation] --> INC QUANT[Quantity mutation] --> INC TEMP[Temporal reversal] --> INC REM[Remove sole support] --> INC end Metamorphic testing defines reliability through expected relationships between transformed cases rather than through a single exact reference answer.
One transformation deserves special treatment:
append the candidate claim verbatim to the evidence Containment should improve because the evidence representation now contains the claim.
If provenance says the new passage is circular or untrusted, policy may still reject it.
This is therefore a scope probe, not evidence that the containment measurement failed.
The same metamorphic idea will return in Chapter 9 when we treat sensitivity as a broader reliability signal rather than only an attack technique.
9. Oracle discipline: attack success is conditional on attack validity A red team can accidentally optimize ambiguity instead of vulnerability.
Structural examples are especially vulnerable to this.
Suppose the source says:
A may increase B under some conditions. and the attack says:
A does not increase B. Is that a clean contradiction?
Not necessarily.
The modality makes the label ambiguous.
So structural attacks should prefer oracle-controlled construction.
A structured fact can be represented as:
fact = { "subject": "Company A", "relation": "acquired", "object": "Company B", "date": "2024-03-14", "polarity": "positive", } Then render the supported statement and apply controlled transformations.
Attack Transformation Oracle condition Relation inversion swap subject and object relation declared asymmetric in template Polarity reversal flip polarity original statement explicit and unqualified Quantity mutation replace exact value source establishes one exact value Temporal inversion swap before/after source fixes temporal order Binding recombination pair subject with another object’s relation structured facts define original bindings Support deletion remove sole supporting passage support graph confirms unique decisive edge For every generated attack, record whether it is actually valid.
Define:
$$ \operatorname{ValidityRate} \frac{N_{valid}}{N_{attempted}}. $$
Then report attack success conditional on validity:
$$ \operatorname{ASR}_{valid} \frac{N_{successful;and;valid}}{N_{valid}}. $$
For example, an attack report should be able to distinguish:
10,000 attempted transformations 9,420 oracle-valid examples 6,110 successful detector evasions from a method that produces only 4,000 valid examples out of 10,000 attempts.
The first quantity measures attack generation yield.
The second measures detector vulnerability.
Do not collapse them.
And do not use semantic similarity as the ground-truth oracle for structural attacks.
High semantic similarity is often exactly the property we are trying to preserve while truth changes.
The adversary should attack the detector, not the annotation procedure.
10. Freeze both sides—and keep three adversarial suites separate Adaptive red teaming creates a familiar evaluation trap.
Suppose we:
attack detector v1 observe failure change detector add attack to benchmark attack again change detector again Eventually the detector may achieve 100% on the accumulated failures.
That tells us it remembers known failures.
It does not prove it generalizes adversarially.
So separate three artifacts.
graph TD AD[ATTACK DEVELOPMENT: invent / tune transformations, inspect failures] --> FREEZE[FREEZE ATTACKER] FREEZE --> HOT[HELD-OUT ADVERSARIAL TEST: new examples / entities / sources] HOT --> MIT[MITIGATE] MIT --> REG[KNOWN FAILURE ENTERS REGRESSION] REG --> NOVEL[NOVEL-MECHANISM SEARCH CONTINUES] A failure can graduate into regression coverage only after the attacker is frozen and a held-out adversarial test has separated remembered cases from new pressure.
Known-failure regression suite Question: Did the detector forget a failure we already discovered? Every confirmed counterexample belongs here.
Held-out adversarial generalization suite Question: Does the mitigation generalize to unseen entities, sources and examples of the SAME failure mechanism? The attacker is frozen before this test.
Novel-mechanism red team Question: Can we discover a DIFFERENT blind spot? This is where adaptive exploration belongs.
Record:
detector version attacker version attack-development corpus held-out adversarial corpus query budget random seed oracle method threshold version The principle is worth preserving:
A test stops being a test when its result changes what you try next.
11. Preserve counterexamples as executable evidence about the detector A counterexample is wasted if it appears once in a notebook and disappears.
Store it with the relationship the future detector is expected to preserve.
counterexample = { "attack_id": "binding_swap_0042", "detector": "hallucination_energy", "detector_version": None, "failure_family": "content", "failure_mechanism": "role_binding", "attack_objective": "false_acceptance", "attacker_access": "score_based", "oracle_label": "UNSUPPORTED", "oracle_method": "structured_relation_swap", "oracle_version": None, "source_evidence": [], "original_claim": "Company A acquired Company B.", "adversarial_claim": "Company B acquired Company A.", "pre_attack_score": None, "post_attack_score": None, "delta_score": None, "threshold": None, "attack_budget": 1, "edit_distance_tokens": 2, "validity": None, "realism": "synthetic_capability_probe", "expected_relation": "post_attack_risk_should_increase", "expected_policy": "REVIEW_OR_REJECT", "first_compromised_stage": "measurement", "first_failed_version": None, "status": "ACTIVE_REGRESSION", } Possible statuses include:
ACTIVE_REGRESSION MITIGATED ACKNOWLEDGED_LIMITATION Not every proxy boundary should be hidden by increasingly complex patches.
Sometimes the correct response is to record:
this sensor does not measure that property.
The regression artifact should therefore test relationships, not only exact score values.
A new embedder or recalibration may legitimately move absolute scores.
The durable expectation may simply be:
negating the decisive proposition must increase risk or:
consistent fictional renaming should not materially change support score This makes the corpus more robust to future implementation changes.
A red-team report should aggregate these records:
red_team_report = { "victim_version": None, "threshold_version": None, "attacker_version": None, "attack_families": ["content", "context", "configuration_pipeline"], "budgets": [1, 4, 16, 64, 256], "attempted_attacks": None, "valid_attacks": None, "validity_rate": None, "baseline_far": None, "baseline_frr": None, "induced_asr_false_accept": None, "induced_asr_false_reject": None, "held_out_results": None, "first_compromised_stage_counts": None, "regression_ids": [], } That turns adversarial testing into an auditable engineering artifact rather than a collection of anecdotes.
12. The current Hallucination Energy results already show three different failure regimes Our existing experiments — all using the semantic-neighbour hard_mined_v2 negatives, not the energy-aware adversary — contain three qualitatively different outcomes.
Wikipedia / FEVEROUS-style Hard-mined mismatches retained substantial mean separation:
supported mean ≈ 0.3714 hard-negative mean ≈ 0.6950 gap ≈ +0.3236 Containment still carried useful signal.
But strict low-FAR operation accepted few supported examples.
This is primarily an operating-point problem until stronger attacks prove otherwise.
PubMed The hard-negative gap remained similarly positive:
supported mean ≈ 0.3700 hard-negative mean ≈ 0.6944 gap ≈ +0.3244 Again, semantic-neighbour mismatches did not erase the signal.
CaseHOLD The ordering reversed:
supported mean ≈ 0.6680 hard-negative mean ≈ 0.6075 gap ≈ -0.0605 This is a different kind of result. On this relation-heavy legal task the detector inverts: it scores the structurally wrong holdings as more contained, on average, than the correct ones. A threshold cannot fix a sign flip. Chapter 8 §15 explains why.
The hard negatives can look more contained than the supported examples.
Now the question is not:
Which threshold should we choose? It is:
Which distinction is missing from the representation or proxy? That is exactly the result adversarial evaluation should search for:
a regime where proxy and target stop moving in the expected direction.
The historical hard_mined_v2 result is a Level-3 semantic-neighbour attack at $K\le16$. It does not yet exercise the energy-aware selection the Certum code already supports, and it does not yet prove robustness across search pressure.
The next concrete experiments are therefore obvious:
energy-aware mining (hardest_energy_mined) at matched conditions K-budget degradation sweep structured binding / relation corpus support-deletion metamorphic tests false-rejection paraphrase + fragmentation tests rank / segmentation attack matrix frozen-threshold domain transfer Those are research tasks, not numbers we should invent in prose.
13. A compact red-team protocol The chapter can now be reduced to a repeatable procedure.
graph TD F[1 — Freeze and classify: victim version, threshold, security vs stress vs falsification] O[2 — Declare the oracle and attack surface: what makes it supported/unsupported, attacker knowledge, capabilities, budget] B[3 — Establish baseline errors: un-attacked FAR/FRR] P[4 — Apply pressure across three families: content, context, configuration/pipeline] V[5 — Validate every attack: attempted count, validity rate, ASR conditional on validity] H[6 — Freeze the attacker and test held out] A[7 — Attribute, preserve, and continue: first compromised stage, regression artifact, acknowledged boundary] The protocol makes adversarial evaluation reproducible by freezing the victim and attacker, measuring valid attacks, and preserving failures without confusing regression with discovery.
1 — Freeze and classify Declare:
victim version threshold / calibration security threat vs stress test vs scientific falsification 2 — Declare the oracle and attack surface Specify:
what makes the candidate supported / unsupported what the attacker may know what the attacker may change what budget applies 3 — Establish baseline errors Measure the un-attacked FAR/FRR before giving the attacker credit.
4 — Apply pressure across the three families CONTENT role / polarity / quantity / time / binding CONTEXT wrong neighbor / deletion / inflation / echo / duplication / distractors CONFIGURATION + PIPELINE segmentation / rank / embedder / threshold / retrieval / attribution / aggregation Increase search budget where appropriate.
5 — Validate every attack Report:
attempted count oracle-valid count validity rate attack success conditional on validity score effect distribution 6 — Freeze the attacker and test held out Separate attack development from final adversarial evidence.
7 — Attribute, preserve, and continue Record:
first compromised stage propagated consequence regression artifact held-out generalization result acknowledged proxy boundary if not fixable Then start searching for the next mechanism.
The objective is not to finish red teaming.
It is to make the system increasingly explicit about what it can and cannot guarantee.
14. What breaking the detector actually teaches us The purpose of adversarial evaluation is not to prove that every detector is bad.
Any finite sensor can be pushed outside its contract.
The useful result is more precise.
Breaking a detector tells us which statement about it was too strong.
For Hallucination Energy:
WRONG: low energy means true STILL WRONG: low energy means supported CLOSER, BUT TOO STRONG: low energy means the evidence entails the proposition MEASUREMENT-SAFE CLAIM: low energy means the claim embedding is well represented by the configured evidence-derived subspace The final statement survives because it is the actual measurement definition.
The attacks then tell us what must exist around it:
containment + relation / binding checks + polarity / quantity / temporal structure + provenance + verification + policy Sometimes a red team finds an implementation bug.
Fix it.
Sometimes it finds a calibration or pipeline bug.
Repair the system.
Sometimes it finds a boundary in the observable itself.
Do not pretend calibration can recover information the sensor never represented.
When the implementation is correct, adversarial evaluation narrows our interpretation of the measurement until the interpretation matches what the measurement can actually support.
A successful red team turns an overclaim into a boundary.
That is not a defeat for the metric.
It is progress in understanding it.
Research roots This chapter combines our Certum hard-negative work with broader research on dynamic benchmarks, difficult hallucinations, process-level verification and metamorphic testing.
Steven Cho, Stefano Ruberto and Valerio Terragni, “Metamorphic Testing of Large Language Models for Natural Language Processing,” 2025. Surveys 191 metamorphic relations and implements 36 representative relations across roughly 560,000 LLM tests, demonstrating how transformation relations can expose failures when exact output oracles are difficult. https://arxiv.org/abs/2511.02108
Borui Yang, Md Afif Al Mamun, Jie M. Zhang and Gias Uddin, “Hallucination Detection in Large Language Models with Metamorphic Relations,” 2025. Introduces MetaQA, using prompt mutations and metamorphic relations for hallucination detection without requiring external factual resources. https://arxiv.org/abs/2502.15844
Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda and Pascale Fung, “HalluLens: LLM Hallucination Benchmark,” ACL 2025. Includes dynamic test-set generation intended to mitigate leakage and strengthen robustness. https://aclanthology.org/2025.acl-long.1176/
Shrey Pandit, Jiawei Xu, Junyuan Hong, Zhangyang Wang, Tianlong Chen, Kaidi Xu and Ying Ding, “MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models,” EMNLP 2025. Builds 10,000 PubMedQA-derived examples through a controlled hallucination pipeline and reports that harder hallucinations are semantically closer to ground truth and harder for detectors to identify. https://aclanthology.org/2025.emnlp-main.143/
Wenbo Chen, Veena Padmanabhan, Tootiya Giyahchi, Elaine Wong and Leman Akoglu, “Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insights,” ACL 2026. Highlights gaps in realistic long-context RAG evaluation and label-noise stress testing. https://aclanthology.org/2026.acl-long.680/
Yu Zhang, Peter Belcak, Shizhe Diao, Yonggan Fu, Shaona Ghosh, Morteza Mardani, Eileen Margaret Peters Long, Bei Yu and Pavlo Molchanov, “PROBE: PROcess-Based BEnchmark for Hallucination Detection,” Findings of ACL 2026. Decomposes hallucination detection into claim decomposition, evidence finding, evidence evaluation and hallucination localization, enabling stage-level diagnosis. https://aclanthology.org/2026.findings-acl.2099/
Xinyi Li et al., “OpenHalDet: A Unified Benchmark for Hallucination Detection across Diverse Generation Scenarios,” 2026. Standardizes evaluation across diverse generation settings and detector families under black-box, gray-box and white-box access regimes. https://arxiv.org/abs/2606.06959
Ernan Hughes, Certum, open-source implementation of claim–evidence geometry, hard-negative mining, calibration, policy gating and evaluation. The hard_mined_v2 procedure shortlists up to 16 centroid-similar evidence sets and selects the most similar mismatch (with a source-page and element-id leakage guard); a separate hardest_energy_mined mode instead selects the shortlisted mismatch with minimum Hallucination Energy. https://github.com/ernanhughes/certum
Next: Containment Is Not Truth We started Chapter 5 with a geometric hypothesis.
We evaluated it in Chapter 6.
Chapter 7 has now deliberately attacked it under controlled pressure.
The attacks expose a recurring pattern:
all semantic ingredients can be present while the proposition is still wrong A false claim can preserve:
entities topic vocabulary semantic neighborhood evidence-span membership while changing:
role polarity quantity time causal direction binding That is no longer merely an evaluation issue.
It is a statement about what containment means.
Chapter 8 takes that adversarial result and asks the deeper first-principles question:
Why can a claim be completely contained by its evidence representation and still be false?
The answer will force us to separate semantic material from propositional structure—and to stop expecting one scalar to carry information that the representation never encoded.