Selection is a filter, and a filter can only return the best thing that was put into it.

That is a practical limitation rather than a pedantic one, because the candidates come from one model answering one prompt, and such samples correlate. When four candidates share a misreading of the evidence, the selector does not detect the misreading. It ranks four versions of it and reports the winner with a confident score.

Generating more samples may not help when the error is systematic. If repeated samples keep reproducing the same misreading of the evidence, additional breadth creates more versions of the same defect rather than a genuinely new route around it. The failure this chapter addresses is therefore not merely a shortage of alternatives. It is a defect that persists strongly enough that generating another independent answer is an inefficient way to attack it.

The obvious response is to hand the draft back to the model and ask for something better. That produces a loop, and it is worth being precise about why the loop underperforms: an instruction to improve this names no target, so the model is free to satisfy it by changing anything at all.

It usually changes the writing.

Prose gets smoother, hedges get added, length grows, and the underlying mistake is carried forward intact into a version that now reads as though it has been checked.

So the mechanism this chapter adds is not a better rewrite prompt. It is the separation of one vague instruction into three stages that can each be inspected and measured on their own:

A critique is a defect hypothesis. A revision is an intervention against that hypothesis. The acceptance gate decides whether the intervention earned its place.

    flowchart TD
    D[current version] --> C[critique]
    C -->|no material defect| S[stop]
    C -->|defect hypothesis| R[targeted revision]
    R --> E[evaluate old against new]
    E -->|improved| A[accept: new becomes current]
    E -->|not improved| B[reject: restore old]
    A --> D
    B --> S
  

Note that the loop has two exits and only one of them produces a changed answer.

That asymmetry is the whole design. A revision loop without a rejecting exit is a machine for accumulating whatever the reviser happened to do last.

This also gives reflection a definition that does not depend on any claim about model introspection.

Nothing here requires the model to think about its own thinking. It requires a diagnosis, an intervention, and a measurement, and those three artifacts either exist as separate inspectable objects in the runtime or they do not.

1. The rewrite that changes everything except the mistake

An agent is asked to diagnose an intermittent failure. It answers that the API is probably returning null, and recommends adding a null check before parsing the response.

The evidence supplied with the task says something else: the failure appears only under concurrent writes, serial execution never reproduces it, and the stack trace runs through shared mutable state. Every one of those points to a race, and none of them supports a null response.

Ask that agent to improve its answer and you are likely to get a second version explaining that the API may intermittently return null under load, with a recommendation to add defensive validation and more detailed logging. The hedging is new. The reference to load is new.

The diagnosis is identical, and it is still contradicted by all three pieces of evidence.

Nothing in the request forced the model to state what was wrong with the first answer. Without that, the cheapest way to satisfy “improve” is cosmetic, and cosmetic changes are exactly what a fluent model produces most reliably.

The first change is therefore an explicit diagnosis step, before any rewriting happens.

What is the highest-impact defect? What evidence supports that claim? What specifically must change, and what must survive?

A revision has something to act against only once those questions have been answered separately from the rewrite itself.

2. Breadth and depth answer different failures

The candidate-selection chapter searched horizontally, sampling several complete answers and comparing them. This chapter searches vertically, staying on one answer and attempting a directed change.

    flowchart LR
    subgraph breadth [breadth: independent samples]
        P[prompt] --> A1[A]
        P --> B1[B]
        P --> C1[C]
        A1 --> SEL[select]
        B1 --> SEL
        C1 --> SEL
    end
    subgraph depth [depth: one trajectory]
        V0[v0] -->|critique| V1[v1]
        V1 -->|critique| V2[v2]
    end
  

These are not competing philosophies, and the choice between them follows from what the first attempt usually looks like:

The last row is the one most often forgotten.

Best-of-N can often spend its generation compute in parallel, while revision is causally sequential because each revision depends on feedback about the previous version. With sufficient parallel capacity, generating N independent candidates can therefore approach the wall-clock latency of one generation, whereas N dependent revision rounds remain roughly N serial steps. Provider limits, batching, candidate length and evaluator cost can change the actual ratio.

Two conditions should stop you reaching for revision at all.

If objective environment feedback already identifies the fix, apply the fix rather than asking a model to describe it. And if the critic and the evaluator share the same proxy bias, the loop will optimize that bias efficiently and quietly, which is worse than not running it.

The two strategies compose, and later the chapter on trajectory search generalises both by branching over partial trajectories rather than finished answers. For now the input to this chapter is a single candidate that already won a selection.

3. A critique is a defect hypothesis, not a verdict

There is a tempting inference to avoid: the critic says X is wrong, therefore X is wrong.

The critic is another fallible call to the same class of model that produced the draft, so its output has exactly the epistemic status of the draft. Treating it as ground truth imports an error and then acts on it with confidence.

Naming it a hypothesis is not a rhetorical softening.

It changes the code, because a hypothesis needs supporting evidence attached, a severity that can be compared against a threshold, and a downstream test that can disconfirm it. A verdict needs none of those things.

A critique worth acting on therefore carries five fields: what appears wrong, what evidence supports the claim, how serious it is, what should change, and what must survive the change.

from dataclasses import dataclass
from enum import StrEnum


class DefectCategory(StrEnum):
    UNSUPPORTED_CLAIM = "unsupported_claim"
    MISSING_EVIDENCE = "missing_evidence"
    CONTRADICTION = "contradiction"
    CONSTRAINT_VIOLATION = "constraint_violation"
    INCORRECT_REFERENCE = "incorrect_reference"
    VERBOSITY = "verbosity"


@dataclass(frozen=True)
class Critique:
    category: DefectCategory
    target: str
    defect: str
    evidence: tuple[str, ...]
    revision_actions: tuple[str, ...]
    preserve: tuple[str, ...] = ()
    severity: int = 0

    @property
    def defect_key(self) -> tuple[DefectCategory, str]:
        return self.category, self.target

The category field gives defects a stable vocabulary for telemetry and comparison, while target identifies where the defect applies. Free-text defect descriptions are too unstable to use as identities, but category alone is too coarse: two different contradictions can occur in two different parts of the same answer. The pair (category, target) therefore becomes the runtime’s stable defect key. It is coarse enough to compare across revisions and specific enough to avoid treating every defect of one category as the same problem.

For the debugging example, a real critique reads: category CONTRADICTION, target failure diagnosis, defect the answer asserts a null-response failure while the supplied evidence describes a timing-dependent one, evidence failure appears only during overlapping writes, serial execution succeeds, the stack trace crosses shared state, actions remove the null-response diagnosis and explain why the timing evidence indicates a race, preserve the logging recommendation and the 300-word limit.

Compare that to a critic that reports the answer could be improved for accuracy and clarity.

The second version is not a weaker critique. It is not a critique at all, because there is no intervention that follows from it and nothing in it can turn out to be wrong.

4. Prefer evidence the runtime can obtain itself

Before asking a model to invent a criticism, ask whether the defect can be measured. A great many can be, and every one that can be is a defect the runtime should detect for itself rather than delegate to a probabilistic call.

A model reporting that a function probably fails its tests is strictly weaker than a runtime reporting that test_parse_date failed at line 84.

The first is a guess about an observable fact. The second is the fact, and the action boundary established the same rule for a different reason: work the runtime can do deterministically should not be handed to the model.

Model critique belongs in the right-hand column, where the failure resists encoding as a check.

An argument can miss the key evidence, an explanation can be internally inconsistent, an answer can satisfy every stated constraint while violating the intent behind them. Those are real defects and no assertion library detects them.

This is also where the research literature is most useful, because it is unusually direct about the limits.

Self-Refine showed that iterative feedback and refinement from a single model improves output across a range of tasks.[1] Constitutional AI used explicit written principles to drive critique and revision inside a larger training procedure.[2] CRITIC demonstrates a self-correction setup in which critique is grounded through external tools, and its results emphasize the value of external feedback rather than relying only on internal model judgement.[3]

Against those, Huang et al. report that intrinsic self-correction on reasoning tasks can fail to improve performance and can actively degrade it,[4] and Kamoi et al.’s survey separates the cases and concludes that the reliability of the feedback source is what distinguishes them.[5]

Read together, these results suggest an engineering distinction.

External evidence can give critique an independent signal that the original generation did not contain. Tests, retrieved sources, compiler errors and other environment observations can constrain a critique in ways that ungrounded self-evaluation cannot.

The engineering consequence is to keep the evidence source visible in the type, which is why Critique.evidence exists and why an empty evidence tuple is a signal worth logging.

5. One defect at a time, or none at all

Suppose the critic returns eight problems: too verbose, weak evidence, missing edge case, poor naming, unsupported claim, unclear conclusion, bad ordering, missing caveat. Ask a reviser to fix all eight and you have not requested a revision. You have requested a new answer, with the original supplied as inspiration.

The result is uninterpretable in both directions.

If it scores better, you cannot tell which of the eight changes mattered; if it scores worse, you cannot tell which one broke it. The loop has spent a model call and learned nothing that would inform the next one.

Ranking by severity and revising against the single highest-impact defect costs nothing extra and makes every outcome legible.

The default is one material defect, one targeted intervention, one measurement, and one conclusion. Some defects are causally coupled and cannot sensibly be repaired independently. When that happens, group only the minimum set that must change together and treat that set as the intervention being tested.

Rollback becomes trivial because there is exactly one thing to undo.

The critic also needs a way to say that nothing is worth fixing, and it needs to be a distinct value rather than an absence:

class NoMaterialDefect:
    """The critic looked and found nothing worth another revision."""


NO_MATERIAL_DEFECT = NoMaterialDefect()
Verdict = Critique | NoMaterialDefect

Returning None for this would collapse two different situations into one.

A critic that examined the draft and found it sound is a success, and a critic call that failed to parse is an error, and a runtime that cannot distinguish them will report a clean bill of health for a broken component.

The same reasoning produced the tagged rejection type at the action boundary, and it applies here for the same reason: an outcome type should record which thing happened, not just that something did not.

Without this escape hatch, a loop with a budget of three will perform three revisions on an answer that was already correct, because another model call was available and nothing in the code prevented it.

6. Revision is a patch, not a rewrite

Given a defect hypothesis, the reviser’s instruction is to change the current draft in the smallest way that addresses this specific defect while preserving what is listed. A revision prompt built from a Critique therefore carries the task, the current draft, the defect, the evidence, the required actions, the preserve list, and any global constraints.

The preserve list is the field most often omitted, and it prevents the characteristic failure of these loops.

A reviser told only what is wrong will happily fix it by regenerating the whole answer, discarding three correct sections to repair one incorrect paragraph. The score may even go up, which makes the loss invisible.

Preservation is checkable, so check it:

def preservation_violations(
    old: str,
    new: str,
    preserve: tuple[str, ...],
    holds: Callable[[str, str], bool],
) -> tuple[str, ...]:
    return tuple(c for c in preserve if holds(old, c) and not holds(new, c))

The holds predicate is supplied by the caller, because what counts as preserving an API signature differs from what counts as preserving a word limit.

Note the double condition. A property that the old version already violated is not a preservation regression, so the reviser should not be blamed for losing something that was already absent. But preservation constraints and hard output requirements are different.

• A preserve rule asks: did something that previously held get broken?
• A hard requirement asks: does the proposed version satisfy this condition now?

The acceptance gate should check both when both exist. Only a constraint that held before and stopped holding afterwards is evidence that this revision broke something. That gives the loop a rejection reason that has nothing to do with scores. A revision that fixes its target defect, improves its evaluation, and destroys the API signature is a failed revision, and the gate should say so in those terms.

7. A revision must earn its place

A loop that always keeps the newest version is a regression machine.

The arithmetic is unforgiving. Start at 0.84, let the critic find a genuine weakness, let the reviser fix it while introducing two smaller problems, and arrive at 0.76. Repeat three times.

Rollback has to be part of the algorithm rather than an operator’s intervention. Every revision is a proposed patch, and the current version stays current until something displaces it.

def accept_revision(old: Candidate, new: Candidate, *, margin: float) -> bool:
    if old.verified_success != new.verified_success:
        return bool(new.verified_success)
    if old.score is None or new.score is None:
        return False
    return new.score - old.score >= margin

This gate decides between two Candidate values carried forward from the selection chapter, which matters for more than type compatibility.

A candidate has both a score, which is a proxy produced by an evaluator, and a verified_success, which is the environment’s report about what actually happened. The first branch says that when the two disagree, the proxy loses.

Consider what the alternative ordering would do.

A revision that makes the test suite pass while scoring lower on a rubric would be rejected in favour of a version that reads better and does not work.

Ordering the checks this way applies the previous section’s principle to acceptance rather than diagnosis: use the strongest available evidence, and fall back to the proxy only when nothing stronger exists.

The margin parameter is not decoration, and the next section explains where its value comes from.

8. The evaluator is a component, not an oracle

Calling something an evaluator does not make it correct, and two failure modes matter enough to measure directly.

The first is bias.

If the evaluator rewards length, the reviser will discover that without any weights changing, because the acceptance gate is a selection pressure and the reviser is generating candidates against it. Every accepted revision is one the evaluator preferred, so a loop run against a verbose-favouring evaluator converges reliably on verbosity.

Nothing in the system is broken, and the output gets worse each round.

The second is noise.

If repeated evaluations of the same text vary by roughly 0.02, then a rise from 0.840 to 0.845 is not evidence of improvement, and accepting it is accepting a coin flip.

Rather than choosing the margin blindly, measure repeated-score variation and use it to establish a practical noise floor:

def evaluator_margin(
    evaluate: Callable[[str, str], float],
    task: str,
    samples: list[str],
    *,
    repeats: int = 5,
    k: float = 2.0,
) -> float:
    noise = [
        statistics.pstdev([evaluate(task, s) for _ in range(repeats)])
        for s in samples
    ]
    return k * max(noise)

This is a heuristic calibration, not a statistical confidence guarantee. It measures repeatability under the evaluator configuration being tested. It does not detect systematic bias, and a deterministic evaluator can have zero repeat variance while still being consistently wrong.

Scoring the same unchanged text several times isolates evaluator variance from genuine differences between drafts, and taking the worst case across samples avoids calibrating on the evaluator’s easiest input. The gate then requires an improvement larger than the instrument can manufacture on its own.

Where absolute scoring is poorly calibrated, a pairwise comparison can be easier to interpret because the task remains fixed and only the two versions change. It is not automatically more reliable. The previous chapter showed that pairwise judges can exhibit order effects and cyclic preferences, so the same controls still apply. Where an objective environment signal exists, it outranks both.

9. Three roles, three debugging surfaces

The same underlying model can serve all three calls. The roles should still be separate calls, because a single prompt asking a model to review an answer, improve it, and report whether it is now better returns one blob of text in which three independent failures are indistinguishable.

    flowchart TD
    D[current version] --> C[critic<br/>which defect?]
    C --> R[reviser<br/>what change?]
    R --> E[evaluator<br/>did it help?]
    E --> G{acceptance gate}
    G -->|accept| N[new current version]
    G -->|reject| O[restore previous]
    C -.measured by.-> M1[detection rate<br/>false-positive rate]
    R -.measured by.-> M2[adherence<br/>preservation rate]
    E -.measured by.-> M3[noise<br/>rank agreement]
    classDef model fill:#e8eef7,stroke:#4a6fa5
    classDef runtime fill:#f2f2f2,stroke:#888
    class C,R,E model
    class G,N,O runtime
  

The critic and reviser are typically model calls. The evaluator is a separate component whose implementation may be deterministic, learned or model-based. The important boundary is therefore not three models feeding a gate. It is:

    fallible diagnosis
    → fallible intervention
    → independent evaluation
    → deterministic acceptance policy

Keeping those responsibilities separate gives each one its own debugging surface.

Consider two systems that both produce a bad revision.

In the first, the critic targeted the wrong defect and the reviser executed the instruction faithfully. In the second, the critic identified the real defect and the reviser changed something else.

Both look identical in an end-to-end score, and they need opposite fixes: better critique prompting for one, tighter revision constraints for the other. A single metric called reflection failed cannot tell you which you have.

Using different models for different roles may reduce correlated blind spots, and it also adds cost, latency, and another configuration axis. Hold the task set and the evaluator fixed and measure whether it helps, rather than assuming that heterogeneity is free improvement.

10. Termination, oscillation, and the budget

The chapter on runtime state handles progress and stopping properly. This chapter needs only enough control to keep the loop finite and make its stopping reason explicit. Six useful termination conditions are:

That table is a type rather than documentation, for the same reason the action boundary made its rejection stages an enum:

class Termination(StrEnum):
    NO_MATERIAL_DEFECT = "no_material_defect"
    LOW_SEVERITY = "low_severity"
    REPEATED_DEFECT = "repeated_defect"
    PRESERVATION_BROKEN = "preservation_broken"
    NOT_IMPROVED = "not_improved"
    BUDGET_EXHAUSTED = "budget_exhausted"

Only the last of these is a limit rather than a diagnosis.

A run that ends any other way has told you something about its components. A system that frequently reaches budget_exhausted is worth investigating. It may indicate that the critic continues finding material defects, that successful revisions expose further defects, or simply that the configured budget is too small for the task distribution.

Oscillation deserves particular attention because it looks like activity.

A run that reports verbosity, then an unsupported claim, then verbosity again, then the unsupported claim again is not converging on anything. Each revision reintroduces the defect the previous one removed, and the loop will spend its entire budget trading one for the other.

Detecting this is why Critique carries both a category and a target. The category provides a closed vocabulary; the target distinguishes separate instances of the same defect class. Comparing recent defect_key values gives the runtime a cheap recurrence signal without pretending that every defect in one category is identical. This is working state that lives and dies with a single computation, not memory; the chapter on selective recall deals with information that must survive beyond the call that produced it.

A budget of three is not a universal constant.

What matters is that unbounded revision is not a design, and that the number is stated somewhere a reader can find it.

11. The loop, and why its history is the result

Everything above assembles into one function. The types it operates on come from the previous chapter, and the helpers do the interesting work, so the loop itself is mostly a sequence of named exits.

from dataclasses import dataclass, replace
from enum import StrEnum
from typing import Callable
import statistics

@dataclass(frozen=True)
class Revision:
    version: int
    candidate: Candidate
    critique: Critique | None
    accepted: bool


@dataclass(frozen=True)
class RevisionRun:
    task: str
    history: tuple[Revision, ...]
    termination: Termination

    @property
    def final(self) -> Candidate:
        return [r.candidate for r in self.history if r.accepted][-1]

    @property
    def defect_path(self) -> tuple[DefectCategory, ...]:
        return tuple(r.critique.category for r in self.history if r.critique)

    @property
    def net_gain(self) -> float | None:
        first, last = self.history[0].candidate, self.final
        if first.score is None or last.score is None:
            return None
        return last.score - first.score

Rollback is a property rather than a branch.

final returns the last candidate that passed the gate, so a rejected proposal stays in the history as evidence without ever becoming the answer, and no separate restore step can be forgotten.

def improve(
    task: str,
    start: Candidate,
    critique_fn: Callable[[str, Candidate], Verdict],
    revise_fn: Callable[[str, Candidate, Critique], Candidate],
    evaluate_fn: Callable[[str, str], float],
    verify_fn: Callable[[str, str], bool | None] | None,
    holds: Callable[[str, str], bool],
    *,
    budget: int = 3,
    min_severity: int = 3,
    margin: float = 0.03,
) -> RevisionRun:
    current = start
    history = [Revision(0, current, None, True)]
    seen: list[tuple[DefectCategory, str]] = []

    def stop(reason: Termination) -> RevisionRun:
        return RevisionRun(task, tuple(history), reason)

    for version in range(1, budget + 1):
        verdict = critique_fn(task, current)
        if isinstance(verdict, NoMaterialDefect):
            return stop(Termination.NO_MATERIAL_DEFECT)
        if verdict.severity < min_severity:
            return stop(Termination.LOW_SEVERITY)
        if verdict.category in seen[-2:]:
            return stop(Termination.REPEATED_DEFECT)
        if verdict.defect_key in seen[-2:]:
            return stop(Termination.REPEATED_DEFECT)

        seen.append(verdict.defect_key)

        proposed = revise_fn(task, current, verdict)
        if preservation_violations(
            current.value, proposed.value, verdict.preserve, holds
        ):
            history.append(Revision(version, proposed, verdict, False))
            return stop(Termination.PRESERVATION_BROKEN)

        verified = (
            verify_fn(task, proposed.value)
            if verify_fn is not None
            else None
        )

        scored = replace(
            proposed,
            score=evaluate_fn(task, proposed.value),
            verified_success=verified,
        )

        accepted = accept_revision(current, scored, margin=margin)

        history.append(Revision(version, scored, verdict, accepted))
        if not accepted:
            return stop(Termination.NOT_IMPROVED)
        current = scored

    return stop(Termination.BUDGET_EXHAUSTED)

The reason RevisionRun returns a history rather than a string is that a revision loop can fail in every interesting way while still returning a plausible final answer.

Reading the output tells you nothing about whether the critic found real defects, whether the reviser acted on them, whether the gate caught a regression, or whether the loop simply oscillated until its budget ran out.

The trajectory tells you all four, and it costs a tuple to keep.

12. Planted defects measure the critic and the reviser separately

Naturally occurring defects come without labels, which makes critic quality hard to assess and easy to assess badly. Reading five critiques and judging them thoughtful measures their fluency. Construct a controlled benchmark instead. Take answers that are correct, introduce exactly one known error into each, and record which category it belongs to. This benchmark measures whether the components can detect and repair known defect classes under controlled conditions. It does not establish how often those defects occur naturally or whether the synthetic corruptions match the full distribution of real failures.

@dataclass(frozen=True)
class PlantedDefect:
    clean: str
    corrupted: str
    category: DefectCategory


def score_critic(
    cases: list[PlantedDefect],
    critique_fn: Callable[[str, Candidate], Verdict],
    task: str,
) -> CriticScore:
    detected = right = alarms = 0
    for case in cases:
        verdict = critique_fn(task, Candidate(case.corrupted))
        if isinstance(verdict, Critique):
            detected += 1
            right += verdict.category is case.category
        if isinstance(critique_fn(task, Candidate(case.clean)), Critique):
            alarms += 1
    return CriticScore(len(cases), detected, right, len(cases), alarms)

Running the critic against the clean version as well as the corrupted one is what makes the benchmark discriminating, and CriticScore exposes detection rate, category accuracy, and false-positive rate as properties over those counts.

A critic that responds to everything with needs more detail scores a perfect 1.0 on detection and a catastrophic 1.0 on false positives, which is precisely the diagnosis a detection-only benchmark would hide.

The same benchmark isolates the reviser, by bypassing the critic entirely.

Hand the reviser the corrupted draft together with the known defect, its evidence, the required change, and the preserve list, then ask three questions with checkable answers: did the specified defect go away, did the preserved constraints survive, and did anything else break?

That yields adherence, preservation rate, and regression rate for the reviser alone. Without this separation a weak critic makes a strong reviser look useless, and the fix gets applied to the wrong component.

13. The ablation that settles the question

Individual metrics say whether a component works.

An ablation says whether the mechanism was worth building, and it is the experiment most worth running here because each stage of this chapter can be removed independently.

Hold the model, the task set, the sampling settings and the evaluator fixed, then record verified success, quality score, regression rate, model calls, latency, and cost per successful task across all five rows.

The comparisons that matter fall out of adjacent rows.

Row two against row three answers whether structured diagnosis beat a generic rewrite instruction. Row three against row four answers whether the acceptance gate actually prevented regressions, which is the claim most likely to be assumed rather than tested. Row four against row five answers whether a second and third revision bought enough to justify their serial latency.

Those are narrow questions with numeric answers, and they replace the claim that reflection makes agents better.

Given how directly the literature contradicts that claim in some settings,[4][5] a system that cannot show its own ablation has no grounds for making it.

14. What we earned

The selection chapter could rank what a model happened to produce. This chapter can change it, under conditions strict enough that the change is attributable: one diagnosed defect, one targeted intervention, one measurement against the version it proposes to replace, and a rejecting exit that allows the runtime to preserve the previous version whenever the intervention fails to earn its place..

That is a small mechanism, and it buys one specific thing. Improvement stops being a property we hope emerges from asking nicely and becomes an event the runtime can confirm, refuse, and explain afterwards. It also exposes the next limitation clearly. Every mechanism so far assumes a candidate already exists and asks how to judge or repair it, which works when the task is answerable in one shot and fails completely when it is not.

An agent asked to migrate a schema, reconcile two datasets, or fix a bug spanning four files does not produce a flawed answer worth critiquing. It edits before it inspects, performs step four before step two, and rediscovers at step six a constraint it should have settled at the start.

No critique loop repairs that, because the defect is not in the answer.

It is in the order of the work.

Research roots

This chapter is an engineering reconstruction rather than a survey, and the references below are selective. Each is cited because the mechanism here depends on something it established, and the two critical results are as load-bearing as the three positive ones.

1. Madaan et al. — Self-Refine: Iterative Refinement with Self-Feedback (NeurIPS 2023). Generates an output, obtains feedback from the same model, and refines repeatedly; the source for the basic claim that an iterative feedback loop can improve results.
   https://arxiv.org/abs/2303.17651

2. Bai et al. — Constitutional AI: Harmlessness from AI Feedback (2022). Uses explicit written principles to produce critiques and revisions inside a larger training procedure, which is the precedent for giving the critic a stated standard rather than a general instruction to improve.
   https://arxiv.org/abs/2212.08073

3. Gou et al. — CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (ICLR 2024). Routes critique through external tools rather than internal judgment, which is the direct source for preferring evidence the runtime can obtain itself.
   https://arxiv.org/abs/2305.11738

4. Huang et al. — Large Language Models Cannot Self-Correct Reasoning Yet (ICLR 2024). Reports that intrinsic self-correction without external feedback can fail to improve reasoning and can degrade it, which is why this chapter’s loop rejects by default.
   https://arxiv.org/abs/2310.01798

5. Kamoi et al. — When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs (TACL 2024). Separates the forms of self-correction and identifies feedback reliability and experimental design as what distinguishes the successful cases from the rest.
   https://arxiv.org/abs/2406.01297

Next: Planning and Execution

The mechanisms built so far all operate on a finished candidate. They assume the shape of the work was obvious enough that the model could attempt it in one pass, which holds for a surprising number of tasks and fails badly for the rest.

The next chapter represents the work itself before any of it is done, separating what the agent intends to do from what it is doing now.

The difficulty is that a plan is another artifact a model produced, so it needs the same treatment everything else has received: an explicit representation, a boundary between proposing and executing, and a way to notice when the plan has stopped matching the world it was written for.