A plan is a claim about the future. It says that reproducing the failure, then diagnosing it, then patching, then testing, is a route from here to a working system. Making that claim explicit was worth the machinery. But it is written before any of the work happens, and execution may invalidate one of its assumptions almost immediately.

Execution produces something else entirely: a trajectory. Actions went out, observations came back, and some of what came back may contradict the intended route. The patch step failed because a dependency is missing. The test step is not merely late; it is unreachable.

A runtime that cannot represent this difference has no way to notice that the future it planned and the state it actually reached have diverged, and therefore no principled way to decide what should happen next.

That gap is what this chapter closes.

An agent loop is reliable only when the runtime can tell whether the trajectory is making progress, and can stop for a reason it can name.

The action boundary asked whether a single proposal may execute, in isolation, with no memory of anything that came before it.

The question here is temporal and cumulative: given everything that has already happened, should anything execute at all?

1. The loop that cannot stop

Here is the shape almost every agent starts as:

while True:
    action = decide(observation)
    observation = execute(action)

It is a loop, but it is not yet a runtime.

There is no representation of what is currently true, what has already been tried, whether anything improved, what remains, what has been spent, or why the run should end. The final line means repeat forever unless the model happens to decide otherwise, which hands the model authority over a decision it is poorly placed to make.

Consider what the software already knows at the moment the model is asked to continue: seventeen actions have executed, four of them proposed identically; no required artifact has changed; three tool calls failed with the same error code; forty-two seconds and $1.37 are gone.

None of that requires linguistic judgment. These are execution facts, and the runtime has them exactly, for free, in a form it can compare against a threshold.

Asking a model to infer them from a transcript converts arithmetic into interpretation.

That is a bad trade, and the rest of the chapter is about not making it.

2. Runtime state is not the transcript

The first implementation of state is nearly always the message list:

messages = [
    {"role": "user", "content": task},
    {"role": "assistant", "content": response},
]

This is useful model context and a poor system of record.

The runtime cannot ask it whether the failing test has been reproduced, or whether the target file changed, without re-reading prose and hoping the answer is in there. What the runtime needs is a structured working record it can query directly, keyed to the plan it is executing:

from dataclasses import dataclass, field


@dataclass
class RunState:
    goal: str
    requirements: "Requirements"
    completed: set[str] = field(default_factory=set)   # PlanStep ids
    failed: set[str] = field(default_factory=set)
    facts: dict[str, object] = field(default_factory=dict)
    artifacts: set[str] = field(default_factory=set)
    trace: list["Transition"] = field(default_factory=list)
    blocked_attempts: set[str] = field(default_factory=set)
    model_calls: int = 0
    cost: float = 0.0
    recoveries: int = 0
    stop_reason: "StopReason | None" = None

    @property
    def step(self) -> int:
        return len(self.trace)

    @property
    def steps_after_last_progress(self) -> int:
        for i, transition in enumerate(reversed(self.trace), start=1):
            if transition.progress.productive:
                return i - 1
        return len(self.trace)

completed and failed hold PlanStep identifiers, so the plan built in the previous chapter and the state built in this one refer to the same objects.

That is the difference between two mechanisms that compose and two that merely coexist.

The word state still needs one caution. RunState is not reality itself. It is the runtime’s explicit working representation of what execution has established so far. Its quality depends on the observations and reducers that update it. The advantage is not omniscience; it is that control-relevant facts no longer have to be reconstructed from prose on every turn.

The published ICLR 2025 version of ActionReasoningBench evaluates reasoning about actions and change across eight planning domains with action sequences up to nineteen steps. It reports average accuracy of 65.63% on state tracking and 58.73% on action executability, despite giving the model the relevant action sequence.[2] It also finds degradation as sequences grow, persistent difficulty with properties that actions do not change, and a systematic disadvantage on negative fluents.

Those findings do not prove that one particular RunState design is correct. They do establish the engineering problem: asking a model to repeatedly reconstruct evolving and non-evolving facts from a growing history is itself a fallible computation.

A fact the runtime established once can instead remain explicit until an observation changes or withdraws it.

Likewise, "dependency_install_failed" can remain represented as a negative execution fact rather than relying on the model to remember a sentence saying that an earlier attempt did not work.

The wider question of what should survive beyond this run — what a system remembers across tasks, weeks or users — belongs to the chapter on memory.

Here we need only the working state of one execution.

3. State comes from observations, not narration

Suppose the agent runs a focused test suite and the environment returns 3 failed, 12 passed. A weak runtime asks the model to summarise that and then treats the summary as truth.

A stronger one records the observation first and derives state from it through an explicit reducer.

The action boundary already gives us the observation type, Observation(ok, kind, data). What this chapter adds is the reducer that relates an accepted action and its observation to the current state, and the trace that preserves the sequence:

@dataclass(frozen=True)
class Transition:
    step: int
    fingerprint: str          # action identity, used for cycle detection
    attempt_key: str          # state + action identity, used for recovery
    action: dict
    observation: Observation
    progress: "Progress"

The interleaved action-and-observation trajectory is not a new idea; ReAct made this pattern central to modern LLM-agent interaction.[1] What changes here is that it stops being only a prompting pattern and becomes data the runtime owns, with an explicit type, an index and an append-only discipline.

The snapshot and the trace answer different questions. The snapshot answers what is represented as true now: three tests are failing. The trace answers how we got here: seven failures at step 4, three at step 8, three at step 12, three at step 16. The snapshot alone cannot distinguish a difficult problem from a stalled one; the sequence can.

    flowchart LR
    A[action] --> E[environment]
    E --> O[Observation<br/>ok, kind, data]
    O --> R[reduce<br/>state + action + observation]
    R --> S[RunState<br/>facts, artifacts, completed]
    R --> T[trace<br/>append-only Transitions]
    S --> V[model_view<br/>projection]
    T --> V
  

A model may still help interpret an ambiguous observation, such as an unfamiliar stack trace or a warning whose significance depends on context.

What it must not be is the only place an execution fact exists. If a fact is important enough to determine whether another action may run, it should have an explicit runtime representation.

4. Progress is a relation between states, not activity

An agent can search, read, edit, run, search and edit again for twenty steps without moving. Every one of those steps produces output, consumes budget and looks purposeful in a log. None of it is necessarily progress, because progress is not a property of an action.

It is a property of the pair of states an action sits between.

What counts as movement is domain-specific, and stating it concretely is more useful than defining it abstractly:

The tempting implementation is a single float, after.score - before.score. Sometimes that is enough. More often the task has several dimensions that trade against each other, and collapsing them early destroys exactly the information needed to decide what to do next. A run that fixed two tests while introducing a regression is not the same as one that fixed two tests cleanly.

@dataclass(frozen=True)
class Progress:
    task_delta: float
    evidence_delta: float = 0.0
    regression_delta: float = 0.0
    integrity_ok: bool = True
    measurement_ok: bool = True

    @property
    def productive(self) -> bool:
        if not self.measurement_ok or not self.integrity_ok:
            return False
        moved = self.task_delta > 0 or self.evidence_delta > 0
        return moved and self.regression_delta <= 0

evidence_delta means task-relevant evidence that narrows uncertainty or unlocks a justified next step, not merely more logs or more text. regression_delta keeps damage visible rather than letting one improving dimension hide another worsening one.

measurement_ok is equally important. If the runtime lacks the facts needed to measure progress, that is not the same thing as measuring zero progress. The controller should preserve that distinction rather than turning missing instrumentation into a plateau.

A scalar can still be derived later if something genuinely needs one. For loop control, the structured signal is more useful.

5. A progress signal without integrity constraints is a target

Define progress as the number of failing tests decreases, and the following looks superb:

8 failures → 0 failures

until you discover the agent deleted the test file. Define it as URLs visited and a research agent can improve without limit while answering nothing. This is Goodhart’s law arriving on schedule: a useful proxy becomes dangerous when the policy can optimize the proxy without preserving the property we actually care about.

The fix is not merely a better metric. It is a metric plus the constraints that keep the metric meaningful, evaluated together.

def measure_repair(before: RunState, after: RunState, obs: Observation) -> Progress:
    failing_before = before.facts.get("tests_failing")
    failing_after = after.facts.get("tests_failing")
    required = after.facts.get("required_tests")
    present = after.facts.get("tests_present")
    new_failures = after.facts.get("new_failures")

    if any(
        value is None
        for value in (
            failing_before,
            failing_after,
            required,
            present,
            new_failures,
        )
    ):
        return Progress(task_delta=0.0, measurement_ok=False)

    return Progress(
        task_delta=float(failing_before) - float(failing_after),
        regression_delta=float(new_failures),
        integrity_ok=set(required) <= set(present),
    )

The previous version of this example defaulted missing test counts to zero and missing required-test sets to empty. That is convenient code and dangerous measurement: absence of instrumentation can masquerade as success. Here, missing inputs make the progress measurement explicitly unusable.

When the required tests are actually absent, integrity_ok is what prevents an apparent 8 → 0 improvement from becoming productive progress.

The same shape applies elsewhere: claim coverage can rise while citation support falls; fields completed can rise while a previously valid field is overwritten; latency can fall because a required check was skipped.

Establishing that success has genuinely occurred is a larger problem, and it gets its own chapter. What this chapter needs is the warning that follows from the structure:

Every continuation and stopping decision inherits the trustworthiness of the progress signal it consumes.

6. Repetition, cycles, and plateaus

Identical actions are trivial to recognize. Fingerprint the structured proposal:

import hashlib
import json


def fingerprint(action: dict) -> str:
    payload = json.dumps(action, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(payload.encode()).hexdigest()[:16]


def attempt_key(state_key: str, action: dict) -> str:
    payload = f"{state_key}:{fingerprint(action)}"
    return hashlib.sha256(payload.encode()).hexdigest()[:16]

The two fingerprints deliberately answer different questions.

fingerprint(action) asks whether the same action shape is recurring, which is useful for cycle detection.

attempt_key(state_key, action) asks whether the runtime is attempting the same action from materially the same state. That distinction becomes important during recovery: RUN_TESTS may be a useless repeat before a code change and exactly the right action after one.

The application supplies state_key because what makes two states materially equivalent is domain-specific. For a coding task it might include the current diff hash, failing-test set and relevant plan position. It should exclude bookkeeping that changes without changing the problem — model-call count, elapsed cost, recovery count — or every retry will appear novel. A key that ignores control-relevant state can over-block or under-block recovery.

Catching repetition is not the same as stopping on it. Plenty of tools are meant to be called repeatedly, among them poll_build_status, scroll_down and read_next_page.

Repetition is evidence of a loop only when paired with the absence of progress.

The harder case is that real loops are rarely A → A → A. They are A → B → A → B, or edit → test → inspect three times over, which is also precisely what productive iterative work can look like.

    flowchart TD
    C["edit → test → edit → test"] --> Q{any productive<br/>transition in the cycle?}
    Q -- yes --> W[iterative work<br/>continue]
    Q -- no --> L[nonproductive cycle<br/>stop or recover]
  

That distinction is implementable directly:

def repeating_cycle(trace: list[Transition], max_period: int = 2) -> int | None:
    """Period of a repeating action pattern in the tail of the trace, if any."""
    marks = [t.fingerprint for t in trace]

    for period in range(1, max_period + 1):
        window = period * 2
        if len(marks) < window:
            return None

        tail = marks[-window:]
        if all(tail[i] == tail[i % period] for i in range(window)):
            return period

    return None


def nonproductive_cycle(state: RunState, max_period: int = 2) -> bool:
    period = repeating_cycle(state.trace, max_period)
    if period is None:
        return False

    repeated_tail = state.trace[-period * 2:]
    return not any(t.progress.productive for t in repeated_tail)

Period 1 recovers the identical-action case. Period 2 catches the A → B → A → B oscillation the naive check misses.

Plateaus need a different measurement, because a plateau involves no repetition at all: every action can be unique and irrelevant. The RunState.steps_after_last_progress property walks back to the last productive transition.

def stalled(state: RunState, patience: int) -> bool:
    return state.steps_after_last_progress >= patience

This turns the agent seems stuck into five consecutive executed transitions produced no task progress and no useful new evidence. That is a statement the runtime can log, test and use as policy.

7. The continuation policy

State and progress make continuation an explicit decision with an owner. The model may propose continuing, finishing, retrying or replanning.

The runtime decides, using the same boundary the action chapter established: proposal on one side, authority on the other.

    flowchart TD
    P[model proposes] --> V{requirements complete<br/>and externally confirmed?}
    V -- yes --> S[stop: SUCCESS]
    V -- no --> M{progress signal<br/>usable?}
    M -- no --> U[stop: PROGRESS_UNAVAILABLE]
    M -- yes --> B{budget remaining?}
    B -- no --> X[stop: budget reason]
    B -- yes --> C{nonproductive cycle<br/>or stalled?}
    C -- no --> G[continue]
    C -- yes --> R{safe recovery<br/>available?}
    R -- yes --> RW[restore and retry differently]
    R -- no --> E[stop: RECOVERY_EXHAUSTED]

    classDef model fill:#e8f0fe,stroke:#4285f4
    classDef runtime fill:#fef7e0,stroke:#f9ab00
    class P model
    class V,M,B,C,R,S,U,X,G,RW,E runtime
  

Every bounded episode needs an outer limit, and one counter is not enough, because a step is not a unit of anything. One step reads a file; the next runs a twenty-minute integration suite.

Bound the resources that actually deplete:

@dataclass(frozen=True)
class Budget:
    max_steps: int = 12
    max_model_calls: int = 20
    max_seconds: float = 90.0
    max_cost: float = 1.00
    max_recoveries: int = 2
    no_progress_patience: int = 5
    max_cycle_period: int = 2

    def __post_init__(self) -> None:
        if self.max_cycle_period < 1:
            raise ValueError("max_cycle_period must be at least 1")
        if self.no_progress_patience <= 2 * self.max_cycle_period:
            raise ValueError(
                "no_progress_patience must exceed twice max_cycle_period "
                "so cycle detection gets a chance to name the failure"
            )

The last two values are coupled. Detecting a period-2 cycle requires four transitions; if patience is 3, the plateau check fires first and the run stops with NO_PROGRESS before the oscillation becomes visible.

The earlier version stated that relationship as advice. Here the policy object enforces it.

A budget is not a quality measure. It is a bound on how much the system may spend before returning control. General AgentBench provides useful contemporary evidence for keeping sequential horizons under scrutiny: performance can initially improve with additional interaction but then plateau or degrade once accumulated history crosses a model- and domain-dependent context ceiling.[3]

More steps do not reliably buy more capability, so a generous step budget is not automatically safer.

When a run ends, the reason is data:

from enum import StrEnum


class StopReason(StrEnum):
    SUCCESS = "success"
    MAX_STEPS = "max_steps"
    MAX_MODEL_CALLS = "max_model_calls"
    TIME_BUDGET = "time_budget"
    COST_BUDGET = "cost_budget"
    NO_PROGRESS = "no_progress"
    NONPRODUCTIVE_CYCLE = "nonproductive_cycle"
    PROGRESS_UNAVAILABLE = "progress_unavailable"
    RECOVERY_EXHAUSTED = "recovery_exhausted"
    USER_INPUT_REQUIRED = "user_input_required"
    UNRECOVERABLE_ERROR = "unrecoverable_error"

Collapsing these into the agent failed throws away the most actionable thing the run produced:

One proposal deserves particular suspicion. When the model emits {"action": "finish"}, that is a claim about the world made by the component whose output is under evaluation. Setting status = SUCCEEDED on that claim alone would make the producer the sole judge of its own product.

The runtime checks its own completion requirements and calls an external confirmation seam instead. A refused finish still consumes whatever model usage produced it, so repeated premature finish proposals eventually hit policy bounds rather than spinning for free.

8. Recovery has to change something

Retry repeats an attempt under materially the same conditions.

Recovery changes a condition that could plausibly alter the result: different arguments, a different tool, a revised plan, a restored environment, a fresh model context, or an escalation to the user. The distinction matters because a runtime that detects no progress and responds by replanning into the same plan, which proposes the same action, which fails the same way, has merely moved the loop up one level.

For long runs there is a second problem. A bad early action can alter both the model’s context and the environment, and later forward actions may not be able to reconstruct the state that existed before the mistake.

AgentRewind, released as a 2026 preprint, is built around this recovery problem. It records aligned checkpoints of agent context and controlled environment state so execution can return to an earlier state and resume with information learned from an abandoned attempt.[5] The word aligned is the substantive part.

Restoring runtime state without restoring the corresponding environment, or restoring the environment without the runtime state that described it, creates a combination that never existed.

The checkpoint interface therefore has to represent both sides:

from copy import deepcopy
from typing import Callable


@dataclass(frozen=True)
class Checkpoint:
    label: str
    step: int
    state: RunState
    environment_token: object


class RecoveryError(RuntimeError):
    pass


def rewind(
    cp: Checkpoint,
    current: RunState,
    restore_environment: Callable[[object], bool],
) -> RunState:
    if not restore_environment(cp.environment_token):
        raise RecoveryError("environment checkpoint could not be restored safely")

    revived = deepcopy(cp.state)

    # Usage and recovery counters describe the whole episode, not the branch.
    revived.model_calls = current.model_calls
    revived.cost = current.cost
    revived.recoveries = current.recoveries + 1

    # Block only the exact state+action attempts abandoned on this branch.
    revived.blocked_attempts = current.blocked_attempts | {
        transition.attempt_key
        for transition in current.trace[cp.step:]
    }

    return revived

A checkpoint object must contain a snapshot rather than a live reference to RunState; a take_checkpoint implementation should deep-copy the state when it captures the corresponding environment token.

Three counters deliberately do not rewind. Model calls, cost and recovery attempts were spent even if the branch is later abandoned. Restoring them would make recovery a way to erase its own budget consumption.

The blocked set is now state-aware. The earlier implementation carried raw action fingerprints forward, which was too coarse: a test command that was useless before a patch may be exactly the right command after the patch. Blocking the attempt_key instead refuses only an action from materially the same runtime state.

This safety depends on the application-defined state key being meaningful. If the key omits the state change that makes an action worth retrying, the runtime can over-block. If it changes for irrelevant reasons, it can under-block.

Irreversible side effects remain a hard boundary. A checkpoint cannot un-send an email, un-charge a card or un-merge a branch unless the surrounding environment provides a real compensation or restore mechanism. restore_environment must refuse rather than pretend otherwise.

    flowchart LR
    CP[aligned checkpoint<br/>runtime + environment] --> S1[attempt A]
    S1 --> S2[attempt B]
    S2 -.->|no progress| R[restore checkpoint]
    S1 -.-> K[blocked attempt keys]
    S2 -.-> K
    K -.-> R
    R --> ALT[different attempt]
  

Take checkpoints at moments of meaningful, safely restorable progress rather than on a blind step counter.

A checkpoint captured mid-plateau restores the runtime to a state that is already stalled. A checkpoint that cannot restore the corresponding environment is not a checkpoint the controller may safely use.

9. Represent the distance still to go

Binary success is too coarse for long tasks.

A ten-requirement engineering job that satisfies eight and then fails is recorded as “failure”, discarding the fact that eighty per cent of the work is done and specifically which twenty per cent is not.

@dataclass(frozen=True)
class Requirements:
    ids: frozenset[str]
    satisfied: frozenset[str] = frozenset()

    @property
    def remaining(self) -> int:
        return len(self.ids - self.satisfied)

    @property
    def complete(self) -> bool:
        return self.ids <= self.satisfied

Requirements.satisfied is runtime state, not a model declaration. The reducer should update it only from observations or checks that establish a requirement, so a model saying done cannot move an item from remaining to satisfied by itself.

Long-horizon benchmarks have converged on the value of preserving partial completion rather than only a terminal bit. Odysseys evaluates 200 realistic long-horizon web tasks and annotates each with an average of 6.1 graded rubrics, reporting better agreement with human judgement than trajectory-level LLM judging.[4] MettleBench, introduced with AgentRewind, likewise scores partial checklist progress on long engineering assignments alongside task completion.[5]

Odysseys adds a second measure worth borrowing directly. Trajectory efficiency is rubric score per step, progress divided by the cost of obtaining it, and frontier agents reach only 1.15% on it against a 44.5% success rate.[4]

The gap between those two numbers is the argument of this chapter in a single comparison: agents that eventually succeed are spending most of their trajectory not succeeding, and a runtime that only measures the endpoint cannot see it.

10. What the model sees is a projection

The runtime may keep a large trace and a rich state object. The model does not need all of either, and contemporary evidence on sequential agents gives us reason to be cautious about feeding accumulated history back indefinitely.[3]

Between runtime state and model context sits a projection:

def model_view(
    state: RunState,
    fact_keys: frozenset[str],
    window: int = 4,
) -> dict:
    return {
        "goal": state.goal,
        "step": state.step,
        "remaining": state.requirements.remaining,
        "completed": sorted(state.completed),
        "failed": sorted(state.failed),
        "facts": {
            key: state.facts[key]
            for key in fact_keys
            if key in state.facts
        },
        "recent": [
            {
                "action": t.action,
                "ok": t.observation.ok,
                "kind": t.observation.kind,
                "productive": t.progress.productive,
            }
            for t in state.trace[-window:]
        ],
    }

The important change is that even facts is projected. A dictionary can be structured and still be enormous, irrelevant or inappropriate to send to a model. The application names the control-relevant fact keys explicitly.

completed and failed preserve execution facts the model should not have to rediscover from a long transcript. productive is a runtime judgement already made from before/after state. Recent transitions give local context; the complete trace remains available to debugging, evaluation and recovery without automatically becoming inference context.

The division of labour is deliberate:

runtime keeps what must remain inspectable and correct
        ↓
projection selects what this decision needs
        ↓
model receives only that working view

A projection can still be badly designed. Too little context hides required facts; too much recreates the transcript problem. That makes the projection itself a component worth testing rather than an excuse to dump state wholesale.

11. The loop controller

Everything above assembles into a controller. The application supplies the parts that are genuinely application-specific: how to decide, how to execute, how observations reduce into state, how progress is measured, what counts as externally confirmed completion, what state differences matter for recovery, and how the controlled environment is checkpointed.

The controller owns continuation policy and accounting.

A budget only works if the components report the resources they consume, so usage is explicit:

from dataclasses import dataclass
from time import monotonic
from typing import Callable


@dataclass(frozen=True)
class Usage:
    model_calls: int = 0
    cost: float = 0.0


@dataclass(frozen=True)
class DecisionResult:
    action: dict
    usage: Usage = Usage(model_calls=1)


@dataclass(frozen=True)
class ExecutionResult:
    observation: Observation
    usage: Usage = Usage()


def account(state: RunState, usage: Usage) -> None:
    state.model_calls += usage.model_calls
    state.cost += usage.cost

This matters because the earlier controller incremented model_calls itself but never updated cost, making max_cost a policy field with no mechanism behind it. Returning usage also lets a tool implementation report nested model calls if execution itself uses a model.

The loop then becomes:

_RECOVERABLE = frozenset({
    StopReason.NO_PROGRESS,
    StopReason.NONPRODUCTIVE_CYCLE,
})


@dataclass
class AgentLoop:
    decide: Callable[[dict], DecisionResult]
    execute: Callable[[dict], ExecutionResult]
    reduce: Callable[[RunState, dict, Observation], None]
    measure: Callable[[RunState, RunState, Observation], Progress]
    verified: Callable[[RunState], bool]
    take_checkpoint: Callable[[RunState], Checkpoint | None]
    restore_environment: Callable[[object], bool]
    state_key: Callable[[RunState], str]
    model_fact_keys: frozenset[str]
    budget: Budget

    def run(self, goal: str, requirements: Requirements) -> RunState:
        state = RunState(goal=goal, requirements=requirements)
        checkpoint = self.take_checkpoint(state)
        started = monotonic()

        while True:
            reason = self._stop_reason(state, started)

            if reason in _RECOVERABLE and checkpoint is not None:
                if state.recoveries < self.budget.max_recoveries:
                    try:
                        state = rewind(
                            checkpoint,
                            state,
                            self.restore_environment,
                        )
                    except RecoveryError:
                        state.stop_reason = StopReason.UNRECOVERABLE_ERROR
                        return state
                    continue

                reason = StopReason.RECOVERY_EXHAUSTED

            if reason is not None:
                state.stop_reason = reason
                return state

            decision = self.decide(
                model_view(
                    state,
                    fact_keys=self.model_fact_keys,
                )
            )
            account(state, decision.usage)
            action = decision.action

            if action.get("name") == "finish":
                if state.requirements.complete and self.verified(state):
                    state.stop_reason = StopReason.SUCCESS
                    return state
                continue

            mark = fingerprint(action)
            key = attempt_key(self.state_key(state), action)

            if key in state.blocked_attempts:
                continue

            before = deepcopy(state)

            execution = self.execute(action)
            account(state, execution.usage)
            observation = execution.observation

            self.reduce(state, action, observation)
            progress = self.measure(before, state, observation)

            state.trace.append(
                Transition(
                    step=state.step,
                    fingerprint=mark,
                    attempt_key=key,
                    action=action,
                    observation=observation,
                    progress=progress,
                )
            )

            if progress.productive:
                checkpoint = self.take_checkpoint(state) or checkpoint

    def _stop_reason(
        self,
        state: RunState,
        started: float,
    ) -> StopReason | None:
        b = self.budget

        if state.requirements.complete and self.verified(state):
            return StopReason.SUCCESS
        if state.step >= b.max_steps:
            return StopReason.MAX_STEPS
        if state.model_calls >= b.max_model_calls:
            return StopReason.MAX_MODEL_CALLS
        if state.cost >= b.max_cost:
            return StopReason.COST_BUDGET
        if monotonic() - started >= b.max_seconds:
            return StopReason.TIME_BUDGET

        if state.trace and not state.trace[-1].progress.measurement_ok:
            return StopReason.PROGRESS_UNAVAILABLE

        if nonproductive_cycle(state, b.max_cycle_period):
            return StopReason.NONPRODUCTIVE_CYCLE
        if stalled(state, b.no_progress_patience):
            return StopReason.NO_PROGRESS

        return None

Several details are load-bearing.

measure receives both the immutable before snapshot and the mutated after state, so progress is genuinely relational.

reduce receives the action as well as the observation, so the state transition does not have to recover the identity of the attempted action from prose hidden inside the observation.

Usage makes cost and nested model calls part of the runtime’s accounting interface rather than side effects the controller hopes somebody increments.

Blocked recovery attempts are keyed by state plus action, while cycle detection continues to use action fingerprints. The two mechanisms therefore no longer confuse same action with same attempt.

Finally, started is not reset by a rewind. Wall-clock time is spent whether or not a branch is later abandoned.

One boundary remains intentionally incomplete: verified is still only a seam. If verification itself is expensive or probabilistic, its attempts and budgets must be accounted for too. The later verification chapter owns that policy.

The loop has stopped being a prompt that happens to repeat.

It is software with named state, named exits, explicit accounting and a testable continuation policy.

12. Test loop control as its own subsystem

Loop-control bugs are discovered in production by default because the conditions that trigger them are rare, expensive and non-deterministic. They do not have to be.

Every branch above can be forced with scripted decisions, observations, usage and progress in milliseconds, with no live model or tool in the loop.

B and C are the pair that matters most for cycle detection because they contain the same action pattern and must produce opposite decisions. Any detector that passes one and fails the other is measuring repetition instead of nonproductive repetition.

I and J are the corresponding pair for recovery. A raw action blacklist cannot satisfy both: it either allows the exact failed attempt again or blocks a legitimate retry after the world changed. State-aware attempt identity is what separates them.

K protects the progress machinery from a quieter failure. Missing instrumentation should stop as missing instrumentation, not silently become five steps of apparent non-progress.

L protects the checkpoint abstraction itself. A runtime that cannot restore the environment it claims to rewind must refuse recovery rather than manufacture a state that never existed.

13. Measure the stops you should not have made

A stopping policy can be too aggressive as easily as too permissive, and the aggressive failure is harder to see: the run terminates, the log records NO_PROGRESS, and nobody discovers that one more justified step would have completed the task.

Both directions need measuring.

Run the same task set through a ladder of runtimes, each adding one mechanism:

Record verified success, partial progress, false-positive stops, false-negative loops, model and tool calls, wall-clock time, cost per successful task, and the reason distribution for unsuccessful runs. The ablation matters more than a single heroic configuration because it tells you which mechanism paid and which was decoration.

steps_after_last_progress is especially useful on failed runs. If the final useful advance occurred at step 5 and the run stopped at step 28, then twenty-three executed transitions happened after the last measured improvement. That does not prove every later action was worthless, but it tells you exactly where to inspect the trajectory.

The continuation policy reads the same progress structure that evaluation later analyzes, which keeps the diagnosis coherent. It does not make the metric automatically correct. Section 5 already established why: a policy can optimize a weak progress proxy, and an evaluation can reward the same weakness.

So measure the controller against outcomes it does not own:

progress / stall metrics
        +
false-stop rate
        +
verified task success
        +
integrity failures

A stopping rule is good when it spends less after useful progress has ended without cutting off trajectories that would have earned real success.

14. What this buys

Two mechanisms now describe time from opposite ends. The plan describes intended future work; the state and trace describe what actually occurred. Between them sits a policy that reads both, plus the latest observation, recent progress and remaining budget, and returns one of three answers: continue, recover, or stop with a reason.

The agent is meaningfully more adaptive for it, because future behaviour now depends on evidence accumulated during execution rather than only on a route written before that evidence existed. It is also, and not coincidentally, a good deal more ordinary as software: named types, an append-only trace discipline, a reducer, explicit accounting, and a continuation policy with a test suite.

The increase in capability arrived as a decrease in mystery.

Three boundaries are deliberate and worth stating, because a chapter that annexes its neighbours ends up being about everything. The trace here is local execution history, not memory: nothing survives the run, and deciding what should is a separate problem. verified is a seam, not a verifier: the runtime insists that something external confirm success, and says nothing about what evidence would be sufficient. And the action surface is still assumed rather than designed.

That last one is the more urgent gap.

Every mechanism in this chapter operates on actions the runtime receives, and none of it helps if the actions available are ambiguous, overlapping, dangerously broad or badly described. A loop with excellent control over a bad tool surface fails precisely, repeatedly, and for a reason it can name.

Research roots

This book is an engineering reconstruction rather than a survey, and the references below are selective. They are here to locate the ideas in the literature and to supply evidence for specific claims, not to suggest that any one paper defines the architecture above.

1. Yao et al. — ReAct: Synergizing Reasoning and Acting in Language Models (ICLR 2023). Interleaves model reasoning, actions and environment observations — the interaction pattern this chapter turns into an inspectable runtime trajectory rather than leaving only in model context. https://arxiv.org/abs/2210.03629

2. Handa et al. — ActionReasoningBench: Reasoning about Actions with and without Ramification Constraints (ICLR 2025). Evaluates reasoning about actions and change across sequences of up to nineteen actions, including state tracking and action executability; cited here for the observed degradation with longer action histories and the difficulty models show with static and negative fluents. https://arxiv.org/abs/2406.04046

3. Li et al. — Benchmark Test-Time Scaling of General LLM Agents (2026). General AgentBench reports an effective context ceiling under sequential scaling, beyond which additional interaction turns produce instability rather than improvement; cited for the claim that step budgets should be tight rather than generous. https://arxiv.org/abs/2602.18998

4. Jang et al. — Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks (2026). Introduces graded rubric evaluation for 200 long-horizon web tasks and a trajectory-efficiency metric of rubric score per step; the reported 1.15% efficiency against a 44.5% success rate is the empirical case for measuring the trajectory and not only the endpoint. https://arxiv.org/abs/2604.24964

5. Zhuang et al. — AgentRewind: Recoverable Execution for Long-Horizon LLM Agents (2026). A runtime recovery framework using aligned checkpoints of agent context and controlled environment state; its MettleBench benchmark evaluates task completion and partial checklist progress, supporting both the checkpoint boundary and the partial-progress representation used here. https://arxiv.org/abs/2608.14380

Next: Capabilities and Routing

The runtime can now say what the goal is, what was planned, what was proposed, what executed, what the environment returned, what is true now, whether progress occurred, whether another step is permitted, and why the run ended. What it cannot say is whether the agent should have been able to attempt any of it.

The next chapter designs the action surface itself: what an agent is allowed to do, and how those capabilities should be described and separated so that the right one is selected reliably. Control over time gives way to interface design.