Hallucination on Programmer.ie: Modern AI programming

When Models Make Things Up

Sat, 29 Aug 2026 22:10:00 +0100

A language model can produce an answer that is fluent, grammatical, detailed, confident, and wrong.

That combination is what makes hallucination interesting.

Ordinary software failures usually leave evidence. A parser throws an exception. A database rejects an invalid constraint. A test fails. A network call times out. The failure changes the shape of the result.

A hallucinating language model can do something more difficult to handle: it can fail without looking broken.

Evidence, Truth, and Verifiability

Sat, 29 Aug 2026 23:20:00 +0100

Chapter 2 gave us a rule for talking about hallucination precisely:

Relative to what reference did the output fail?

That question immediately creates another one.

What counts as a reference strong enough to constrain the answer?

Suppose a model says:

Company A acquired Company B in 2024.

There are several different things we might mean when we ask whether this is acceptable:

What exact proposition has been asserted?

Is that proposition true at the relevant time?

Do we possess evidence for it?

Does that evidence actually support this exact relationship?

Can we identify the source and passage that support it?

Where did that source come from?

Is the source sufficiently reliable and current for this claim?

Is the evidence package sufficient for the action we want to authorize?

Could the claim be verified by this system now?

Has the verification actually been performed?

Would the active policy allow the claim to be published or acted upon?

Those questions are related.

The Memory Contamination Problem

Sun, 30 Aug 2026 18:39:00 +0100

Chapter 13 ended with a strict recovery invariant:

repair
→ new candidate state
→ re-measure
→ re-authorize

and with one additional rule:

candidate in HOLD
→ do not persist as trusted factual memory

That second rule changes the scale of the problem.

A hallucinated sentence displayed once is transient.

The same sentence written into persistent state can survive the conversation that created it.

It can then be:

retrieved
summarized
copied
cited
used as agent experience
used to fill a user profile
inserted into a vector index
fed into another model
used as a future repair source

and eventually appear to the system as if it came from somewhere else.

Building Systems That Distrust Their Models

Sun, 30 Aug 2026 21:01:00 +0100

The first chapter began with a simple observation:

A language model can produce a fluent answer
without possessing a mechanism that proves the answer is true.

Fourteen chapters later, that fact has not changed.

The model can still:

invent
misbind
misattribute
ignore decisive context
answer without enough evidence
accept bad retrieval
repair one error by creating another
repeat its own stored mistake

The final architecture does not make those possibilities disappear.