# Advanced Agents From First Principles 06: Does One Agent Plan, Execute and Judge Its Own Work? Build a Planner-Executor-Critic Architecture
# Book: Advanced Agents From First Principles
# Chapter URL: https://programmer.ie/books/advanced-agents-from-first-principles/06-chapter/
# Prose fingerprint (sha256, normalized): 5c254481749d33b1cc93493c617804d471c69ccd16bddda9a13be902c2efa9cc
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

Does One Agent Plan, Execute and Judge Its Own Work? Build a Planner-Executor-Critic Architecture A single model can often do all of these things:
understand a task, decide what to do, execute a tool call, inspect the result, critique its own work, decide whether it succeeded, and produce the final answer. That is convenient.
It is also a dangerous concentration of responsibilities.
If the same component creates the plan, executes it, explains why the result is good, and decides whether the job is complete, then failures become difficult to localize.
A bad plan can look like bad execution.
Bad execution can be rationalized by the critic.
A weak critic can approve the executor’s own mistakes.
A stale plan can continue driving the system even after the environment has changed.
And a model that has already invested ten steps in one trajectory has a strong tendency to interpret the latest result as progress rather than admit that the trajectory is wrong.
The next useful architecture is therefore not simply:
more agents It is:
separate responsibilities A common pattern is:
goal | v planner | v structured plan | v executor | v environment | v evidence | +--------+--------+ | | v v critic verifier | | +--------+--------+ | v continue / revise / replan / stop This is often described as a planner-executor-critic architecture.
But the important idea is not the names.
The important idea is that each component has a distinct contract.
The planner proposes a route. The executor performs actions. The critic diagnoses defects. The verifier decides whether reality supports completion.
Those are different jobs.
And they should remain different even if all four roles happen to use the same underlying model.
The Search Problem This Post Solves People building agents commonly run into questions like:
Why does my AI agent keep changing its plan? Why does my planner create steps the executor cannot perform? Why does my executor ignore the plan? Why does my critic always agree with the generated answer? Why does my multi-agent system keep passing work back and forth? Why does my coding agent say the task is complete when tests still fail? Should I use separate planner and executor agents? Do I need a critic agent? Should the planner and executor use different models? How do I stop planner-executor loops? How do I know whether a critic agent is actually helping? These are not primarily prompting questions.
They are control-system questions.
1. Start With the Failure, Not the Architecture Do not begin with:
I want a planner agent, an executor agent, a critic agent, a reviewer agent, a verifier agent, and a coordinator agent. Begin with a measured failure.
Suppose a simple agent has a verified success rate of 68%.
You inspect failed trajectories and discover:
31% bad decomposition 26% execution mistakes 18% premature success claims 14% tool-selection errors 11% other Now the architecture has a reason to change.
A planner may target decomposition failure.
A critic may target execution defects.
A verifier may target premature success claims.
The architectural question becomes:
Which separated responsibility removes which measured failure? That is much stronger than:
Would multi-agent be better? 2. The Simplest Baseline: One Agent Does Everything Consider a coding agent.
The simplest implementation might look like this:
async def solve(task, model, tools): state = {"task": task, "history": []} for step in range(20): response = await model.decide(state) action = parse_action(response) if action.kind == "finish": return action.answer result = await tools.execute(action) state["history"].append({ "action": action, "result": result, }) raise RuntimeError("step budget exhausted") This can work surprisingly well.
It should remain the baseline.
The problem is that several responsibilities are hidden inside model.decide():
understand plan choose action interpret result revise strategy judge completion When the system fails, all of those responsibilities are entangled.
3. Separate Planning From Execution The first separation is usually:
planner | v plan | v executor The planner does not execute tools.
The executor does not silently redesign the objective.
That gives us two inspectable contracts.
A Structured Plan Avoid prose-only plans like:
First inspect the repository, then probably update the file, then run some tests and fix anything that looks wrong. Instead represent the plan as data.
from dataclasses import dataclass, field from typing import Literal StepStatus = Literal[ "pending", "running", "passed", "failed", "blocked", "skipped", ] @dataclass class PlanStep: id: str objective: str action_family: str depends_on: list[str] = field(default_factory=list) expected_evidence: list[str] = field(default_factory=list) status: StepStatus = "pending" @dataclass class Plan: goal: str version: int steps: list[PlanStep] Now the runtime can inspect the plan independently of the model.
It can ask:
Are IDs unique? Do dependencies exist? Is the dependency graph acyclic? Does each step request an available capability? Does every terminal path have a verification step? Is the plan too large for the budget? These are ordinary software checks.
4. The Planner Contract The planner should not own the entire system.
Its responsibility is narrower:
input: goal capabilities known constraints current verified state output: candidate plan That is all.
A useful interface might look like:
from typing import Protocol class Planner(Protocol): async def create_plan( self, *, goal: str, capabilities: list[str], facts: dict, ) -> Plan: ... The planner does not get to:
grant itself capabilities, fabricate successful execution, redefine the user’s goal, claim completion, or bypass verification. The runtime owns those boundaries.
5. Validate Plans Before Execution A planner can hallucinate steps just like an agent can hallucinate tools.
For example:
1. inspect repository 2. call deploy_to_production 3. run hidden integration test 4. merge PR But perhaps the runtime only exposes:
read_file search_code edit_file run_tests Then the plan is invalid before execution begins.
A simple validator:
class PlanValidationError(Exception): pass def validate_plan(plan: Plan, capabilities: set[str]) -> None: ids = [step.id for step in plan.steps] if len(ids) != len(set(ids)): raise PlanValidationError("duplicate step IDs") id_set = set(ids) for step in plan.steps: unknown = set(step.depends_on) - id_set if unknown: raise PlanValidationError( f"step {step.id} has unknown dependencies: {sorted(unknown)}" ) if step.action_family not in capabilities: raise PlanValidationError( f"unsupported action family: {step.action_family}" ) The model proposes.
The runtime validates.
That rule survives every architecture in this series.
6. The Executor Contract The executor has a different job.
It receives one actionable unit of work and attempts it using the available environment.
@dataclass class StepResult: step_id: str success: bool evidence: dict error: str | None = None class Executor(Protocol): async def execute_step( self, *, step: PlanStep, state: dict, ) -> StepResult: ... The executor should not silently rewrite:
"Run tests" into:
"Skip tests and declare success because the patch looks correct." If execution cannot satisfy the planned step, it should report that explicitly.
For example:
StepResult( step_id="run_tests", success=False, evidence={ "command": "pytest -q", "exit_code": 1, "failing_tests": [ "test_repository_loader", "test_plan_validation", ], }, error="2 tests failed", ) That result becomes evidence for the next control decision.
7. Executor Drift One of the most common planner-executor failures is executor drift.
The planner says:
inspect failing tests The executor decides:
rewrite the implementation The planner says:
edit one file The executor changes twelve files.
The planner says:
perform a read-only diagnostic The executor performs a side effect.
This is not necessarily malicious or irrational.
It often happens because the executor is given an underspecified objective and enough autonomy to reinterpret it.
Detecting Executor Drift Every step should define an execution envelope.
@dataclass class ExecutionEnvelope: allowed_tools: set[str] allowed_paths: set[str] max_tool_calls: int side_effects_allowed: bool Then runtime checks can reject actions outside the envelope.
def authorize_action(action, envelope: ExecutionEnvelope): if action.tool not in envelope.allowed_tools: raise PermissionError("tool outside execution envelope") if action.side_effect and not envelope.side_effects_allowed: raise PermissionError("side effect not allowed for this step") Again:
model autonomy != runtime authority 8. Why Add a Critic? Suppose execution completes.
The result may still be poor.
A coding patch can compile but violate the architectural requirement.
A research summary can cite sources but misrepresent the main claim.
A browser workflow can fill every field but put the wrong data in one field.
A support agent can resolve the ticket operationally while violating policy.
This is where a critic can help.
But only if its responsibility is concrete.
9. A Critic Is Not a Generic “Reviewer” This prompt is weak:
Review the result and tell me if it is good. It encourages generic approval language.
A better critic has a defect contract.
@dataclass class Critique: defect_type: str severity: str evidence: list[str] affected_requirement: str | None suggested_action: str | None confidence: float And the critic receives explicit criteria.
Check only for: - violation of requested behavior - unhandled failure paths - regression risk - evidence missing for claimed completion Now the critic’s output can affect control flow.
10. Critics Need Evidence Too A critic saying:
The implementation looks robust. is weak evidence.
A critic saying:
Potential defect: `load_repository()` catches `Exception` and converts all failures to an empty repository. This can hide authentication and filesystem errors. Evidence: src/repository.py lines 71-78. is more useful.
The critic should point to observable features.
claim + location / evidence + criterion violated That makes the critique inspectable.
11. Critic Agreement Bias A common failure is that the critic agrees with the executor because both share:
the same model, the same prompt context, the same assumptions, the same retrieved evidence, and the same framing. Adding another model call does not guarantee independent judgment.
This architecture:
model A -> executor model A -> critic can easily become:
same failure mode twice The correct question is not:
Did we add a critic? It is:
Did the critic detect additional verified defects? 12. Measure Marginal Critic Value Suppose your baseline executor produces 100 outputs.
External evaluation identifies 40 defective outputs.
A critic detects 25 of them.
That is useful.
Now add a second critic.
If the second critic detects the same 25 defects and only one additional verified defect, its marginal value may be low.
Track:
critic recall critic precision unique verified defects false-positive critique rate cost per additional verified defect A simple record:
@dataclass class CriticMetrics: true_defects: int detected_defects: int false_positives: int unique_defects_vs_previous_critics: int model_calls: int cost: float This prevents critic armies.
13. Critic Is Still Not Verifier This distinction is critical.
critic: "I think this implementation has no obvious defects." is not:
verifier: "All required tests passed against commit abc123." A critic reasons about quality.
A verifier checks externally observable criteria.
The architecture should therefore be:
planner ↓ executor ↓ critic ↓ possible revision ↓ verifier ↓ PASS / FAIL / UNKNOWN Not:
critic says good ↓ success 14. The Verifier Contract The verifier needs explicit success criteria.
from dataclasses import dataclass from typing import Literal VerificationStatus = Literal["PASS", "FAIL", "UNKNOWN"] @dataclass class VerificationResult: status: VerificationStatus evidence: dict state_id: str missing_checks: list[str] For coding agents:
state_id = git tree hash For data systems:
state_id = dataset version For deployments:
state_id = deployment ID For browser workflows:
state_id = transaction / form submission ID Verification must bind to the exact state being accepted.
15. A Full Planner-Executor-Critic Runtime Here is a simplified architecture.
from dataclasses import dataclass, field @dataclass class RunState: goal: str plan: Plan | None = None step_results: list[StepResult] = field(default_factory=list) critiques: list[Critique] = field(default_factory=list) replans: int = 0 revisions: int = 0 class PlannerExecutorCriticRuntime: def __init__( self, *, planner, executor, critic, verifier, max_replans=2, max_revisions=3, ): self.planner = planner self.executor = executor self.critic = critic self.verifier = verifier self.max_replans = max_replans self.max_revisions = max_revisions async def run(self, goal: str, capabilities: set[str]): state = RunState(goal=goal) state.plan = await self.planner.create_plan( goal=goal, capabilities=sorted(capabilities), facts={}, ) validate_plan(state.plan, capabilities) while True: step = next_ready_step(state.plan) if step is None: break result = await self.executor.execute_step( step=step, state=state.__dict__, ) state.step_results.append(result) if not result.success: decision = decide_recovery(state, result) if decision == "replan": if state.replans >= self.max_replans: break state.replans += 1 state.plan = await self.planner.create_plan( goal=goal, capabilities=sorted(capabilities), facts=collect_verified_facts(state), ) validate_plan(state.plan, capabilities) continue break mark_step_passed(state.plan, step.id) critique = await self.critic.review( goal=goal, plan=state.plan, results=state.step_results, ) state.critiques.append(critique) verification = await self.verifier.verify( goal=goal, state=state, ) return state, verification This example is intentionally incomplete.
Production systems need:
authorization, timeouts, tool budgets, cost budgets, retries, concurrency control, state binding, provenance, recovery policies, persistence, failure telemetry, and explicit termination reasons. But the important architecture is visible.
16. The Coordinator Should Mostly Be Code Many multi-agent systems add a coordinator model.
planner ↓ coordinator ↓ executor ↓ coordinator ↓ critic ↓ coordinator ↓ verifier Sometimes that is justified.
Often it is not.
If the routing rule is known:
if plan_missing: call_planner() elif ready_step: call_executor() elif execution_complete: call_critic() elif critique_requires_revision: call_reviser() else: call_verifier() then ordinary code is more reliable than asking another model:
Which agent should speak next? Use model routing only when the choice itself is genuinely semantic or uncertain.
17. Multi-Agent Does Not Require Multiple Models A planner-executor-critic system can use one underlying model.
same model ├→ planner contract ├→ executor contract └→ critic contract The benefit may come from separated state, permissions and objectives rather than model diversity.
Alternatively:
small local model -> planner frontier model -> executor learned scorer -> critic pytest -> verifier Or:
deterministic DAG planner LLM executor static analyzer critic integration test verifier The architecture is about responsibilities.
Not personalities.
18. Planner Drift Planner drift happens when the planner gradually changes the intended objective.
The user asks:
Fix the failing unit test without changing public behavior. After two replans the system is effectively solving:
Refactor the module to make the tests easier to satisfy. That is a different goal.
The runtime should preserve an immutable goal contract.
@dataclass(frozen=True) class GoalContract: objective: str constraints: tuple[str, ...] success_criteria: tuple[str, ...] Replanning can change the route.
It cannot silently change the contract.
19. Stale Plans Plans are hypotheses about the future.
The environment can invalidate them.
Suppose the plan says:
1. edit module A 2. run tests 3. update module B But after step 1, tests reveal module B is unrelated.
Continuing blindly wastes work.
A plan therefore needs replan triggers.
Examples:
unexpected environment state new constraint discovered required tool unavailable critical assumption falsified step output differs from expected evidence verification failure changes diagnosis Do not replan merely because the model feels uncertain.
Tie replanning to observable events where possible.
20. Replanning Without Losing Completed Work A bad implementation throws away the entire plan.
old plan -> failure -> new plan from scratch That can cause repeated work.
Instead preserve verified state.
completed facts completed artifacts successful tool outputs validated constraints Then replan only the unresolved portion.
facts = collect_verified_facts(state) new_plan = await planner.create_plan( goal=goal, capabilities=capabilities, facts=facts, ) This makes replanning incremental rather than amnesic.
21. Circular Self-Approval One subtle failure looks like this:
planner proposes plan executor follows plan critic judges execution against plan Everything can appear internally consistent while the original user goal is still unmet.
Why?
Because the critic is evaluating against the planner’s interpretation rather than the external success criteria.
This is circular self-approval.
The fix is simple conceptually:
user goal ↓ independent success criteria All roles reference that contract.
The planner does not own it.
22. Handoff Loss Every agent handoff can lose information.
planner -> executor executor -> critic critic -> reviser reviser -> verifier If each component receives a prose summary of the previous component, information degrades.
Prefer shared structured state.
@dataclass class SharedTaskState: goal: GoalContract plan: Plan verified_facts: dict execution_results: list[StepResult] critiques: list[Critique] artifacts: dict Then each role reads the same canonical state.
Do not make each model reconstruct reality from another model’s summary.
23. Excessive Handoffs A common advanced-agent failure is architecture inflation.
user ↓ manager ↓ planner ↓ researcher ↓ executor ↓ critic ↓ reviewer ↓ judge ↓ reviser ↓ verifier ↓ manager That looks sophisticated.
It may simply multiply:
latency, token usage, context translation, routing errors, duplicated reasoning, and failure opportunities. Track handoff value.
For each role ask:
What unique failure does this role catch? If the answer is unclear, remove the role and benchmark again.
24. The Failure Matrix A useful way to debug this architecture is to classify failures by responsibility.
Failure Likely subsystem impossible plan planner unsupported tool requested planner / capability contract correct step, wrong execution executor executor changes unrelated files executor authorization repeated unnecessary replan replan policy critic approves clear defect critic critic invents defect critic critic and executor share same blind spot diversity / evidence problem all steps complete but user goal fails goal verifier verification checks old state verifier state binding agents repeatedly hand off coordinator / control policy This is one of the main reasons to separate roles.
Failure attribution becomes possible.
25. Application: Coding Agents Planner-executor-critic architectures fit coding agents well because software already provides strong external evidence.
A possible architecture:
issue / request ↓ planner ↓ repository inspection plan ↓ executor ├→ read files ├→ edit files ├→ run tests └→ inspect diff ↓ critic ├→ architecture violations ├→ regression risk └→ missing edge cases ↓ verifier ├→ tests ├→ type checks ├→ lint ├→ build └→ acceptance criteria Strong planner inputs include:
repository graph, changed files, failing tests, dependency graph, issue constraints, available tools. Strong critic evidence includes:
diff, AST structure, static analysis, test failures, changed dependency boundaries. Strong verification includes:
exact commit/tree hash, test suite results, build artifacts, behavior-level acceptance tests. 26. Application: Code Review A reviewer system may use:
change classifier ↓ risk planner ↓ specialist critics ├→ correctness ├→ security ├→ concurrency └→ API compatibility ↓ external checks ↓ review synthesis Notice that the planner may not be planning implementation work.
It may be planning review coverage.
That is an important application of the same architecture.
27. Application: Research Agents Research agents often fail because one component both forms the hypothesis and evaluates the sources supporting it.
A better split:
research question ↓ planner ├→ subquestions ├→ evidence needs └→ source requirements ↓ executor ├→ search ├→ retrieve └→ extract evidence ↓ critic ├→ unsupported claims ├→ source mismatch └→ contradictory evidence ↓ verifier ├→ source provenance ├→ quote / claim alignment └→ primary-source checks The critic can reason about evidence quality.
The verifier confirms whether the cited source actually supports the claim.
28. Application: Incident Response Incident response is another strong fit.
alert ↓ planner ├→ identify hypotheses ├→ order diagnostics └→ define rollback criteria ↓ executor ├→ inspect logs ├→ inspect metrics ├→ query deployments └→ run read-only diagnostics ↓ critic ├→ alternative root causes ├→ unsafe actions └→ evidence gaps ↓ verifier ├→ service health ├→ error rate └→ rollback / deployment state The side-effect boundary matters enormously here.
Diagnostics can be broad.
Production mutation should remain tightly authorized.
29. Application: Customer Support A support workflow might use:
customer request ↓ planner ├→ identify required information ├→ policy checks └→ resolution path ↓ executor ├→ retrieve account ├→ inspect order ├→ calculate eligibility └→ perform allowed action ↓ critic ├→ policy violation ├→ missing information └→ inappropriate resolution ↓ verifier ├→ refund state ├→ ticket state └→ confirmation ID The verifier checks actual system state.
Not the assistant’s sentence:
Your refund has been processed. 30. Application: Data and Analytics Agents A data agent may split responsibilities like this:
analysis request ↓ planner ├→ required datasets ├→ transformations └→ checks ↓ executor ├→ SQL ├→ transformations └→ statistics ↓ critic ├→ leakage ├→ aggregation errors ├→ denominator mistakes └→ unsupported interpretation ↓ verifier ├→ row counts ├→ schema ├→ constraints └→ reproducible query/output hash Again, role separation maps cleanly onto normal engineering practice.
31. Application: Browser Automation Browser agents often benefit from a planner-executor split because page-level actions are cheap but wrong commitments can become expensive.
objective ↓ planner ├→ page sequence ├→ required fields └→ stop before irreversible submission ↓ executor ├→ navigate ├→ click ├→ type └→ extract ↓ critic ├→ wrong field ├→ unexpected page state └→ missing confirmation ↓ verifier ├→ confirmation page ├→ transaction ID └→ external account state A strong safety rule is:
search and prepare freely commit deliberately verify externally 32. When Should Planner and Executor Use Different Models? Sometimes the planner benefits from a strong model while execution can use a cheaper model.
frontier planner ↓ local executor Sometimes the opposite is sensible.
Planning may be mostly deterministic, while difficult implementation needs the strongest model.
deterministic planner ↓ frontier executor Sometimes both can be local until verification fails.
local planner ↓ local executor ↓ verify ↓ FAIL / UNKNOWN ↓ frontier escalation Benchmark these variants.
Do not assume role prestige maps to model size.
33. Cost-Aware Role Allocation Suppose you have:
local model: cheap, fast frontier model: expensive, stronger static analyzer: deterministic unit tests: deterministic A sensible architecture may be:
local planner ↓ local executor ↓ static critic ↓ tests ↓ if uncertainty / failure remains ↓ frontier critic or executor This is not just multi-agent orchestration.
It is adaptive compute allocation.
34. Planner Confidence Is Not Enough A planner may provide:
{ "confidence": 0.94 } That number is not a reason to trust the plan.
Plan quality should be measured using outcomes.
Track:
plan validity rate steps executed successfully replan rate unnecessary replan rate planner-caused failure rate verified success conditioned on plan type Calibration matters more than confidence prose.
35. Critic Confidence Is Not Enough Either Likewise:
{ "severity": "high", "confidence": 0.97 } may still be wrong.
Measure critic predictions against later verified outcomes.
critic precision critic recall severity calibration false-positive rate marginal defect discovery The critic is a model component.
It should be evaluated like one.
36. Planner-Executor-Critic vs Mixture of Experts These architectures solve different problems.
Mixture of Experts asks:
Who should do this work? Planner-executor-critic asks:
Which responsibility is active now? You can combine them.
planner ↓ router ├→ code executor ├→ research executor └→ browser executor ↓ critic router ├→ security critic ├→ correctness critic └→ policy critic ↓ verifier But do not combine them merely because you can.
Each dimension multiplies complexity.
37. Planner-Executor-Critic vs MCTS MCTS manages search allocation over branches.
Planner-executor-critic manages responsibilities during one trajectory.
A planner can generate candidate strategies that MCTS explores.
An executor can simulate branches.
A critic can provide heuristic value estimates.
A verifier can score terminal outcomes.
But MCTS is not required for planner-executor separation.
Most systems should try the simpler architecture first.
38. Planner-Executor-Critic vs Workflow If the sequence is always:
parse input ↓ query database ↓ apply rule ↓ generate response then use a workflow.
Do not use a planner.
Planner-executor architectures are useful when:
which steps are needed or:
which order should they run changes with the task or environment.
39. A Useful Escalation Ladder Start with:
single model call If execution is fixed:
fixed workflow If next action depends on observations:
agent loop If decomposition fails:
add planner If execution quality fails:
add targeted critic If completion claims are unreliable:
add external verifier If different task types need different capabilities:
add expert routing If multiple trajectories must be explored:
add search That is a much healthier progression than beginning with six agents.
40. Benchmark the Architecture Compare at least:
A. one-shot model B. simple agent loop C. planner + executor D. planner + executor + critic E. planner + executor + critic + verifier For each system measure:
verified success rate planner failure rate executor failure rate critic precision / recall false-success rate replan rate steps model calls tool calls latency cost cost per verified success Then calculate marginal gain.
marginal_gain = ( verified_success_new - verified_success_baseline ) And marginal cost.
marginal_cost = cost_new - cost_baseline Do not merely report that the advanced system used more reasoning.
Report whether it solved more tasks.
41. Role Ablations Ablations are particularly useful here.
Remove the planner.
What changes?
Remove the critic.
What changes?
Replace the LLM critic with deterministic checks.
What changes?
Use the same model for planner/executor versus different models.
What changes?
Remove replanning.
What changes?
This tells you which mechanism is actually earning its keep.
42. Failure Injection Do not wait for natural failures.
Inject them.
For coding agents:
introduce failing test remove required tool return stale file content create conflicting dependency make build pass while acceptance test fails For browser agents:
change element ID insert unexpected confirmation step return partial form state simulate expired session For data agents:
schema drift missing column stale dataset incorrect row count Then measure whether the architecture detects and recovers.
43. Trajectory Logging Advanced systems need trajectory-level observability.
A useful event record:
@dataclass class AgentEvent: run_id: str role: str event_type: str input_state_id: str output_state_id: str | None model: str | None tool: str | None latency_ms: int cost: float metadata: dict Events might include:
plan_created plan_validated step_started step_completed critic_finding revision_requested replan_triggered verification_passed verification_failed Without this, multi-role systems become almost impossible to debug.
44. Handoff Count Is a Metric Track:
handoffs per successful task handoffs per failed task If failures involve far more handoffs, the architecture may be oscillating.
Also track role transitions.
planner -> executor executor -> planner planner -> executor executor -> planner Repeated transitions can expose planner instability.
executor -> critic critic -> executor executor -> critic critic -> executor Repeated transitions may expose unresolved critique loops.
45. Termination Reasons Must Be Explicit Do not return only:
finished = True Use named outcomes.
VERIFIED_SUCCESS VERIFIED_FAILURE UNKNOWN PLAN_INVALID EXECUTION_BLOCKED REPLAN_BUDGET_EXHAUSTED REVISION_BUDGET_EXHAUSTED TOOL_BUDGET_EXHAUSTED TIME_BUDGET_EXHAUSTED AUTHORIZATION_DENIED This matters enormously for production analysis.
46. Do Not Let the Planner Grade the Executor This is tempting:
planner creates expected output executor acts planner decides whether execution matched expectation But the planner may simply defend its own decomposition.
Keep evaluation independent when possible.
Even if independence is only architectural rather than model-level, preserve the contracts.
47. Do Not Let the Critic Rewrite the Goal A critic should identify defects against requirements.
It should not transform:
user requested minimal bug fix into:
rewrite the subsystem because it would be cleaner Preservation constraints matter.
@dataclass(frozen=True) class CriticContract: allowed_defect_types: tuple[str, ...] preservation_constraints: tuple[str, ...] Critique is not authorization.
48. Do Not Let the Executor Treat Critique as Truth A critic can hallucinate.
So the executor should not blindly implement every critique.
A better flow:
critic finding ↓ validate evidence ↓ material defect? ├→ no -> ignore └→ yes -> revise For code, a critic might claim:
this function is unused Before deleting it, search references.
Use tools to test critic claims.
49. Critic-Driven Evidence Acquisition The critic does not need to produce a final judgment immediately.
It can say:
I cannot determine whether this is safe without knowing whether `parse_config()` is used by external callers. That can trigger:
search references inspect public API run compatibility test This is more useful than forcing a binary critique from incomplete evidence.
50. Separate UNKNOWN From FAIL Suppose the verifier cannot access the integration environment.
That is not:
PASS And it is not necessarily:
FAIL It may be:
UNKNOWN The runtime can then decide whether to:
escalate, request human review, retry later, gather more evidence, or stop safely. This distinction prevents false certainty.
51. Real Software Pattern: Compiler Architecture There is a useful analogy here.
A compiler does not ask one giant component to:
parse optimize generate code validate syntax execute program verify behavior It has stages with contracts.
Agent runtimes benefit from the same principle.
intent ↓ plan representation ↓ validated actions ↓ execution ↓ evidence ↓ verification The goal is not to imitate a human team.
The goal is to create inspectable software boundaries.
52. The Architectural Smell Test If your system has agents named:
BossAgent ManagerAgent SupervisorAgent ReviewerAgent JudgeAgent MasterAgent ask what each one uniquely controls.
Names are not architecture.
Contracts are architecture.
A good role definition should answer:
What input does this role receive? What output is it allowed to produce? What authority does it have? What evidence can it observe? What failure does it target? How is its value measured? If those answers are fuzzy, the role probably is too.
53. A Compact Production Architecture A robust default may look like this:
immutable goal contract | v planner | validated plan | v executor | structured evidence | +--------------+--------------+ | | v v deterministic checks critic | | +--------------+--------------+ | revision if justified | v verifier | +-------------+-------------+ | | | v v v PASS FAIL UNKNOWN Notice what is missing:
coordinator LLM Unless routing genuinely requires semantic judgment, ordinary runtime code can coordinate the roles.
54. Production Checklist Before deploying a planner-executor-critic architecture, verify that:
[ ] goal contract is immutable [ ] plans are structured and validated [ ] executor permissions are explicit [ ] executor drift is observable [ ] replanning has concrete triggers [ ] completed verified work survives replanning [ ] critic has a narrow defect contract [ ] critic findings require evidence [ ] critic precision/recall are measured [ ] critic is not treated as verifier [ ] final verification uses external evidence [ ] verification is bound to exact state [ ] UNKNOWN is preserved [ ] handoffs are logged [ ] replan/revision budgets exist [ ] termination reasons are explicit [ ] each role has measurable marginal value If several of these are missing, adding more agents will usually make the system harder to understand rather than more capable.
55. The Evidence Rule The architecture earns its complexity only if it improves verified outcomes.
Suppose:
simple agent verified success: 78% median latency: 8s cost/task: $0.04 And:
planner-executor-critic verified success: 79% median latency: 31s cost/task: $0.22 The advanced system may not be justified.
But if:
planner-executor-critic verified success: 92% median latency: 15s cost/task: $0.09 and the increase comes specifically from fewer decomposition and regression failures, then the architecture has earned its place.
That is the standard.
Not how sophisticated the diagram looks.
56. What We Have Built So Far The advanced series now has several independent mechanisms.
Chain of Thought intermediate computation Self-Consistency repeated sampling + disagreement Tree of Thoughts branching intermediate reasoning MCTS adaptive search-compute allocation Mixture of Experts capability specialization + routing Planner-Executor-Critic responsibility separation + control These are not steps on one capability ladder.
They are dimensions.
A production system may need one of them.
Or several.
Or none.
57. The Next Failure Separating planner, executor and critic creates a new question.
What happens when multiple agents or specialists disagree?
Suppose:
security critic: reject performance critic: accept correctness critic: accept planner: proceed verifier: UNKNOWN Or:
agent A says hypothesis X agent B says hypothesis Y agent C attacks both Simply taking a majority vote can reproduce the self-consistency problem we already saw.
The next architecture therefore needs to ask:
When does adversarial review or multi-agent debate produce genuinely new evidence rather than more correlated model opinions?
That is the subject of the next post.
Conclusion Planner-executor-critic systems are useful because they separate responsibilities that are easy to conflate inside one model call.
The planner decides what should happen.
The executor attempts the work.
The critic looks for specific defects.
The verifier checks whether reality supports success.
The runtime controls the boundaries between them.
The most important lesson is not:
use more agents It is:
Give every responsibility a contract, every handoff a structured state, every claim an evidence path, and every additional role a measurable reason to exist.
That is what turns a collection of prompts into an engineering system.
In the next post we will move from role separation to disagreement itself:
Advanced Agents From First Principles 07: Do Your Agents Agree Too Easily? Use Adversarial Review and Multi-Agent Debate Without Confusing Debate With Truth.