Audit Whether This Agent Separates Planning From Execution
Apply Agents From First Principles Step 04 to a real repository and determine whether plans are explicit, executable, state-aware and independently validated before actions run.
How to use this
- Open a repository-aware AI assistant.
- Give it access to the repository or files you want reviewed.
- Copy the prompt below and run it unchanged first.
- Use the evidence it finds to decide what to inspect or change next.
PromptCopy and run against your own project
You are auditing a repository that implements a multi-step AI agent.
Your job is to determine whether the system has a real planner/executor boundary or merely lets one model improvise while calling the result a plan.
Do not recommend a new planner architecture until you reconstruct the current control flow from code.
Inspect the repository and trace one representative task from user goal to final completion.
Work in this order:
1. Reconstruct the planning boundary
- Find where goals are converted into steps, tasks, dependencies, or a plan object.
- Identify whether the plan is structured data or only natural-language text.
- Record which component owns planning and which component owns execution.
- Identify whether execution can mutate, skip, reorder, or invent plan steps.
2. Build a plan contract
For each planned step record:
- step identifier,
- intended action,
- required inputs,
- prerequisite state,
- expected output,
- completion condition,
- failure condition,
- whether the executor can verify it independently.
3. Trace state between planner and executor
- Determine what repository/runtime state the planner sees.
- Determine what state the executor sees when the step actually runs.
- Find stale-plan risks: changed files, changed remote state, changed tool outputs, changed user intent.
- Identify whether preconditions are checked again before execution.
4. Audit replanning
- Find every path that can trigger replanning.
- Determine whether replanning is based on new evidence or merely model dissatisfaction.
- Check whether already-completed work is preserved or repeated.
- Check whether failed steps can create endless plan-regenerate-plan loops.
5. Audit authority
- Determine whether planning grants execution authority implicitly.
- Identify destructive or high-impact actions that can be planned and then executed without a separate gate.
- Separate: plan proposes -> control permits -> executor acts.
6. Failure injection
Reason through at least these cases:
- a prerequisite step failed but later steps remain runnable,
- repository state changes after planning,
- tool output contradicts a planning assumption,
- executor partially completes a step,
- plan contains an impossible or underspecified action,
- replanning produces a conflicting replacement plan.
Output:
A. Current control-flow trace
B. Planner/executor boundary map
C. Step contract table
D. Stale-plan and replanning failure modes
E. Authority leaks
F. Smallest architectural corrections
G. Concrete tests that prove the corrections
For every finding label it as one of:
- CONFIRMED FROM CODE
- STRONGLY INFERRED
- NEEDS RUNTIME EVIDENCE
Do not treat a model-generated list of steps as evidence that the system has a planner.
A plan is useful only if execution can interpret, constrain, update and verify it as system state.