Use with AI · Chapter Application

Audit Whether This Agent Knows When to Stop

Apply Agents From First Principles Step 05 to a real repository and test whether state, progress signals, budgets and termination reasons prevent unproductive agent loops.

Chapter Application Agents From First Principles Long-running and tool-using agents Intermediate
Read the source chapter → View the learning path →

How to use this

  1. Open a repository-aware AI assistant.
  2. Give it access to the repository or files you want reviewed.
  3. Copy the prompt below and run it unchanged first.
  4. Use the evidence it finds to decide what to inspect or change next.
PromptCopy and run against your own project
You are auditing an AI agent that can iterate, call tools, revise work, or continue until a task is complete.

Your job is to determine whether the loop has explicit progress semantics and defensible stopping conditions.

Do not assume retries are progress.
Do not assume model confidence is completion.

Trace one representative agent run through:
observe -> decide -> act -> observe -> repeat/stop.

Work in this order:

1. Identify loop state
- Find the durable or in-memory state carried between iterations.
- Record what changes after every action.
- Identify counters, budgets, timestamps, completed steps, failed attempts, tool results, and external state references.

2. Identify progress signals
- Find how the system determines whether an iteration improved the task state.
- Separate activity from progress.
- Check whether repeated equivalent actions are detected.
- Check whether the system can recognize no-op edits, identical tool results, unchanged browser state, or repeated failed hypotheses.

3. Identify stopping conditions
Enumerate every stop path:
- verified success,
- hard budget exhaustion,
- retry limit,
- no-progress threshold,
- unrecoverable failure,
- human escalation,
- cancellation,
- timeout,
- model-issued stop.

For each, determine who decides and what evidence is required.

4. Audit budgets
- Token/model-call budget
- Tool-call budget
- wall-clock/runtime budget
- financial cost budget if present
- retry budget per failure class
- search/branch budget if relevant

Check whether nested loops can bypass the outer budget.

5. Audit termination reasons
- Determine whether the final state records WHY the run ended.
- Check whether success, failure, timeout, cancellation and budget exhaustion are distinguishable downstream.
- Check whether callers can accidentally interpret an incomplete run as success.

6. Failure injection
Reason through:
- same tool called with equivalent arguments repeatedly,
- repeated edits toggle code back and forth,
- external state never changes,
- model keeps proposing new wording without new evidence,
- one subtask loops while global budget appears healthy,
- success condition is never externally verified,
- cancellation arrives between action and observation.

Output:

A. Loop state model
B. Progress-signal table
C. Stop-condition table
D. Budget escape paths
E. Infinite-loop / false-progress risks
F. Smallest corrections
G. Tests for no-progress, timeout, cancellation and success termination

Classify findings as:
- CONFIRMED FROM CODE
- STRONGLY INFERRED
- NEEDS RUNTIME EVIDENCE

The key question is not whether the loop has a maximum iteration count.
It is whether the agent can distinguish useful state change from repeated activity.