What Survived the Transformation?

Someone asks a system for a one-line version of the payments retry policy. It answers: “Failed payments are retried up to three times.” The sentence is fluent, plausible and professionally phrased. It is also not what the policy says. The policy had an exception for idempotent requests. It was explicitly uncertain about how a third-party timeout is handled. And it had a source, a design decision made two years ago for a stated reason. The one-liner keeps the headline and loses the rest. Nothing on the screen announces the loss. The sentence looks exactly as good as a faithful one would.

The book has now spent several chapters transforming information. Previews compress pages. Enhancements add diagrams and tables. A record stabilises structure and renderings are generated from it. Sidecars bring in surrounding material, and relations say how one item bears on another. Every one of these can fail silently, because fluent output carries no sign of what happened underneath. This chapter asks the question that covers all of it:

When a representation is transformed, how do we decide whether the transformation preserved what mattered?

The book has circled this since the first chapter, as the question what had to survive? Here it becomes an operating method.

Four outcomes

When a source is turned into something else, four things can be said about any piece of what the receiver might recover from it:

SURVIVAL   something that needed to remain recoverable still is

LOSS       something that needed to remain recoverable no longer is

ADDITION   the transformation introduces a substantive claim, relation,
           implication or confidence not supported by the source

GAIN       the same underlying information becomes easier to recover,
           compare, notice, understand or act upon

The first two are what people usually mean by fidelity. The third is the transformation overreaching. The fourth is the reason to transform at all, and it is the one that tends to go missing from talk about accuracy.

The distinction that matters most is between the last two:

GAIN  ≠  ADDITION

A diagram can make a dependency obvious without inventing a dependency. A preview can make relevance easier to judge without adding a new fact. A table can make a comparison cheaper without changing what the source says. In each case something has improved for the receiver and nothing has been added to the content. Addition is different in kind: a new claim has entered, or an old one has been given a confidence it did not have. The two can look alike on the screen, because both make the representation “more informative”. Telling them apart is a matter of asking what the reader could not have recovered from the source, and whether it is now being asserted.

A fifth case is worth naming, because it is the one fluent generation does best: change. A claim survives as words while its force changes. “May” becomes “is”. “In subgroup Z” detaches from its sentence and generalises. An association acquires an arrow. This is not a separate outcome, because it decomposes into the ones above: the original force can no longer be recovered, which is a loss, and a stronger force has been asserted, which is an addition. Seeing it as both is what makes it detectable.

Declaring what has to survive

None of this can be assessed unless someone says what mattered, and they have to say it before the transformation, not after. Otherwise the judgement drifts to match whatever came out.

BEFORE
declare what must survive
        ↓
TRANSFORM
        ↓
AFTER
check: what survived? what was lost?
       what was added? what became easier?

The declaration can be written in ordinary language. For the retry policy:

must preserve:
- the claim that retries stop after three attempts
- the exception for idempotent requests
- the uncertainty about third-party timeouts
- the source of the policy

That is a preservation obligation. It is a list a person could write in a minute, and its virtue is that it makes disagreement visible. One reader might think the source of the policy is not worth keeping; another might insist on it. Both can now say so, and what survived can be checked against something specific. What had to survive is not a fact about the text alone. It depends on what the representation is for, and so on who will use it for what. The obligation for a quick scan differs from the obligation for someone about to change the policy. Declaring it forces that purpose into the open.

Obligations can themselves be wrong. A list can omit the one thing that mattered. That does not remove the need for them, but it means they too are open to revision, and the revision should be recorded and not silently absorbed.

Content is not function

There is a second way to ask whether something survived, and it matters as much as the first.

CONTENT    did the same facts and relations survive?

FUNCTION   can the recipient still do the thing the representation
           was meant to support?

A transformation can preserve every sentence-level fact and still make the task harder. Reorder a procedure so the steps are all present but scattered, and every claim survives while the person following it can no longer follow it. Bury the one decisive number in the middle of a table and the content is intact while the comparison is gone. In the other direction, a compact representation may leave out a great deal of surface detail and preserve everything the task requires. A summary that drops the background but keeps the decision, the reasons and the caveat may serve a reviewer better than the full text.

So the question is never only whether the words match:

textual sameness  ≠  communicative preservation

This is the first chapter’s point about reconstruction, applied to measurement. The test of a representation is what a receiver can rebuild from it and do with it, so a check that only compares text with text has asked a different question.

A profile, not a score

A single number, such as “fidelity 0.84”, describes none of the ways a transformation can fail, because different transformations fail differently. A diagram tends to preserve topology and lose caveats. A summary tends to preserve claims and weaken uncertainty. A preview tends to preserve topic and exaggerate emphasis. A relation label tends to preserve the entities and reverse the direction. Averaging these into one figure produces reassurance, not information. What is needed is a profile: a short list of things to check, each reported on its own, so that “claims preserved, qualifiers lost” can be seen, attributed and repaired.

A reasonable working list covers the claims themselves, their relations with direction, their qualifiers and scope, numbers, uncertainty, provenance, unsupported additions, and emphasis. Three items in it deserve a closer look.

Relations are first-class. A representation can keep every noun and destroy the assertion. “A causes B only when C is present” becomes “A causes B”: the topic, the entities and part of the relation survive, and the qualifier that governed it has gone. “A acquired B” becomes “B acquired A”: almost every word survives and the meaning reverses. Relation direction, relation type and attached conditions are checked as their own items, because they are where fluent transformations most often fail.

Emphasis can distort without a false statement. Suppose a paper is, by weight of content, mostly method, partly limitation, and a little speculation, and the generated visual gives most of its area and colour to the speculation. No sentence need be wrong. The reader still leaves believing the speculation is the paper. Prominence has been redistributed, and prominence is part of what is communicated. Any system that selects, orders or lays out information is changing emphasis whether or not it states anything false.

Traceability is separate from preservation. A view can be imperfect and still safe, or fluent and dangerous, depending on whether its elements point back to their source. Being able to walk from a statement in the view to the passage that supports it does not make the statement right, but it makes checking and correction possible. An element that cannot be traced is one the reader has to take on trust.

Preservation is also not truth. If a source states something false and the transformation reproduces it faithfully, preservation is high and the claim is still false. Three questions stay distinct: did the transformation preserve what the source said, was what the source said well supported, and is it actually the case? This chapter’s instrument addresses only the first.

The check is an instrument too

There is a trap in everything above. Once there is a list of things to check, it is tempting to treat a check passing as proof that the transformation was faithful. But the check is a measuring device, and it can fail. A test that looks for a phrase can pass because the phrase appears somewhere, while the obligation it stood for, that something be recoverable as what it was, was not met. A scoring method can reward fluency and miss a dropped qualifier.

The way to trust an instrument is to test whether it notices what it should. Take a transformation known to be faithful and remove one decisive element, or alter a number, or reverse a relation, and see whether the check registers the specific loss. If it does not, its pass means nothing. An instrument that has not been shown to discriminate cannot be used to authorise anything.

Where this leaves us

Every transformation can be assessed with the same four outcomes: what survived, what was lost, what was added, and what became easier. Gain is not addition: making existing information easier to use is the point of transforming, while adding unsupported substance is the failure. The obligation has to be declared before the transformation, in plain language, and the result judged both for content and for function, because the same words can be preserved and the task still ruined. The result is a profile, with relations, emphasis and traceability kept in view, and the check that produces it has to be shown to work before it is relied on.

That solves half the problem. A list of what must survive does not tell us whether it did. Someone or something still has to compare the transformed representation with the source, judge each obligation, and decide what counts as preserved. That raises its own questions. Who decides? A person, who is slow and varies? A model, which may share the same blind spots as the system being checked? And when the obligations are written in plain language, how precisely can they be checked at all? That depends on the form the obligation takes, and plain prose is only one of the forms available.

References

This chapter draws on the sources cited in earlier chapters for its examples: the work on spin in compressed summaries of research (the previous discussion of visual compression) and on relation errors in generated diagrams (DiagramEval, EMNLP 2025). Its main concepts are the book’s own and are offered as a method to be tested, not a result.