# Advanced Agents From First Principles 42: How Do Multiple Agents Coordinate Without Becoming a Distributed Argument? # Book: Advanced Agents From First Principles # Chapter URL: https://programmer.ie/books/advanced-agents-from-first-principles/42-chapter/ # Prose fingerprint (sha256, normalized): a46d717690e8de4301737f1093d2688b4427623aa6dc380d14e2010b3b2e5d19 # Snapshot generated at build time by Hugo. This is the complete rendered prose # of the chapter; it is not a transcript of the source file. Multi-agent systems are easy to demo. They are much harder to make correct. Give three models different roles, let them exchange messages, ask them to vote, and the result looks sophisticated. But once those agents can perform real work, the central problem changes. It is no longer: How do we make several agents talk to each other? It becomes: How do several autonomous workers coordinate ownership, evidence, obligations, authority, state and side effects without creating duplicated work, conflicting actions, hidden commitments or persuasive but wrong consensus? That is a distributed-systems problem. And the key principle for this chapter is: Multiple agents should coordinate through explicit contracts and shared state, not by merely chatting until they agree. This chapter builds that architecture from first principles. The Search Problem: “How Should Multiple AI Agents Coordinate?” Most multi-agent examples start with conversation. planner -> coder -> critic -> planner Or: researcher A researcher B researcher C ↓ debate ↓ consensus That is useful for reasoning experiments. It is not enough for production coordination. Suppose three agents are working on a deployment incident. Agent A is investigating logs. Agent B is preparing a rollback. Agent C is validating database state. While they are working: the deployment state changes; Agent A discovers the failure is unrelated to the release; Agent B obtains approval to roll back; Agent C detects that rollback would corrupt a migration; a human changes the objective from “restore previous version” to “stabilize current version”; Agent B still holds a stale execution token; two workers both think they own the rollback commitment. A group chat does not solve this. The system needs explicit answers to questions such as: Who owns the task? Who owns the commitment? Which intent version are we serving? Which state version did each conclusion use? What evidence supports each claim? Who may mutate the external system? Which agent is verifier rather than executor? What happens when agents disagree? What happens when one disappears? What happens when ownership transfers? What happens when two agents act at once? These are coordination semantics. 1. Multi-Agent Reasoning Is Not Multi-Agent Coordination We need a strong distinction immediately. Multi-agent reasoning means several models contribute cognition. Multi-agent coordination means several workers participate in one operational objective. They overlap, but they are not the same problem. A debate architecture may generate better candidate answers without owning any real-world state. A production coordination architecture may have workers that barely communicate in natural language at all. For example: Agent A produces candidate patch Agent B independently runs verification Agent C owns release approval workflow Those agents may exchange structured artifacts rather than conversational messages. That is often preferable. The system should not confuse: communication with: coordination or: agreement with: correctness 2. Start With Shared Intent Identity Every cooperating worker must know which objective it is serving. From Step 37, intent is not merely prompt text. It is versioned control-plane state. A minimal shared identity might look like: @dataclass(frozen=True) class IntentRef: intent_id: str intent_version: int Every delegated task, commitment, candidate, approval and mutation should carry that identity. Why? Because this is unsafe: Agent A serves intent v7 Agent B serves intent v8 Agent C serves intent v7 while all three believe they are cooperating on “the same job.” The natural-language objective may even look almost identical. But if v8 superseded v7, Agent A and Agent C may be performing stale work. So the first multi-agent invariant is: Shared work requires shared authoritative intent identity. Not shared memory. Not shared conversation history. Not “we all received similar prompts.” Exact identity. 3. Decompose Work Before Delegating It An agent should not simply tell another agent: Help me with this. That creates ambiguous responsibility. Instead, delegation should name an explicit unit of work. For example: @dataclass(frozen=True) class WorkAssignment: assignment_id: str intent_ref: IntentRef goal_id: str task_id: str owner_id: str scope: str allowed_actions: tuple[str, ...] required_evidence: tuple[str, ...] completion_contract: str deadline: datetime | None The important fields are not the Python syntax. The important idea is that delegation has boundaries. An assigned worker should know: what it owns; what it does not own; what evidence is required; what counts as completion; what authority is available; which intent version governs the work; whether it owns merely a task or also a commitment. Without that, multi-agent systems tend to create overlapping responsibility. Overlapping responsibility is where duplicate actions begin. 4. Task Ownership Is Not Commitment Ownership Step 38 gave us one of the most important distinctions in the series. A task is executable work. A commitment is a durable obligation. That matters enormously in multi-agent systems. Suppose Agent A owns this commitment: Ensure the production rollback request is either completed or explicitly released before 16:00. Agent A may delegate an API call to Agent B. That does not necessarily transfer the commitment. Agent A owns commitment │ └── delegates task to Agent B If Agent B crashes, Agent A still owns the obligation. By contrast, an explicit commitment transfer would be: Agent A prepare transfer ↓ Agent B acknowledge ownership ↓ control plane advance ownership epoch ↓ Agent A loses authority This should be a structured transition. Not: “You handle it now.” That sentence may express intent. It does not prove operational ownership changed. 5. Ownership Must Be Exclusive Where Effects Are Exclusive Some work may be shared. Some work must have one owner. For consequential mutations, ambiguous ownership is dangerous. Consider: Agent A -> deploy version 4 Agent B -> deploy version 4 If both believe they are responsible, duplicate side effects may occur. We already have the right primitives from Step 20: leases; epochs; fencing tokens; idempotency; authoritative ownership state. Use them here. @dataclass(frozen=True) class OwnershipLease: resource_id: str owner_id: str epoch: int expires_at: datetime A consequential mutation gateway can then require: current intent + current state + valid authority + valid verifier evidence + current ownership epoch before committing. A stale worker may continue thinking. It may even finish generating an excellent action. But it cannot mutate reality with an obsolete epoch. That is exactly what we want. 6. Delegation Should Narrow Authority, Not Expand It Delegation should follow least privilege. If Agent A has authority to: read repository run tests open pull request merge pull request and delegates test execution to Agent B, Agent B should not automatically inherit everything Agent A can do. Its grant might be: read repository snapshot X execute test suite Y write results to artifact store Z Nothing more. A delegation contract should therefore include an authority envelope. @dataclass(frozen=True) class DelegatedCapability: capability: str scope: str expires_at: datetime parent_assignment_id: str The rule is: Delegation transfers only the authority required to perform the delegated work. This prevents authority from expanding transitively across an agent hierarchy. Without it, a harmless research subagent can accidentally become a privileged operational actor. 7. Messages Are Evidence, Not Authority Step 41 established that untrusted content must not become authority simply because a model interprets it as an instruction. The same applies to other agents. Agent B says: I verified the migration is safe. That is a claim. It is not automatically verification evidence. Agent C says: I have approval from the operator. That is also a claim. It is not automatically authorization. Structured messages should carry provenance. @dataclass(frozen=True) class AgentEvidenceMessage: message_id: str sender_id: str assignment_id: str claim_type: str artifact_refs: tuple[str, ...] state_refs: tuple[str, ...] verifier_refs: tuple[str, ...] created_at: datetime The receiver can then interpret the claim while independently validating the referenced evidence. The principle is: Another agent’s statement is evidence about what that agent believes or observed. It is not a privilege escalation path. 8. Shared State Should Be Explicit Many multi-agent demos create one giant shared conversation or shared memory. That is convenient. It is also dangerous. Shared natural-language memory mixes: observations; hypotheses; conclusions; instructions; stale state; plans; commitments; opinions; errors. Production systems need stronger separation. A shared workspace might contain: intent registry assignment registry commitment ledger state snapshots artifact store evidence store operation ledger verification store ownership leases workflow history Agents can then build context from those structured sources. The architecture becomes: SHARED CONTROL STATE intent ─ assignments ─ commitments ─ ownership │ │ │ │ └───────────┴──────┬──────┴─────────────┘ │ artifact/evidence │ ┌─────────────┼─────────────┐ │ │ │ Agent A Agent B Agent C The agents need not share everything. They need shared references to authoritative state. 9. Use Artifact Exchange Instead of Conversation When Possible Suppose Agent A produces a patch. Agent B needs to review it. The robust interface is: artifact_id repository_snapshot base_commit patch_hash candidate_id not: Here’s roughly what I changed… Likewise, a research agent should pass: claim IDs source IDs retrieval timestamps quoted evidence ranges confidence/evidence class rather than a prose summary alone. Natural language remains useful. But structured artifact identity should carry operational meaning. This improves: replay; deduplication; auditing; verification; conflict detection; cache safety; state freshness; incident forensics. 10. Distinguish Collaboration From Independent Verification A classic mistake is to make every agent share its reasoning with every other agent before verification. That creates correlated failure. Suppose Agent A proposes a patch. Agent B is meant to verify it independently. If B receives: A’s full rationale; A’s confidence; A’s interpretation of the tests; A’s desired answer; then B may become anchored. The verifier is no longer independent. A stronger architecture is: Agent A candidate │ ▼ verification gateway │ ├── verifier B receives candidate + specification │ but not persuasive unnecessary rationale │ └── deterministic checks The principle is: Share enough information to verify the artifact, not enough information to manufacture agreement. This is especially important when agents use the same underlying model family. 11. Consensus Is Not Verification Three agents can agree and still be wrong. This should sound familiar from self-consistency. If all agents share: the same model; the same training blind spots; the same prompt; the same retrieved evidence; the same incorrect assumption; then majority vote may simply amplify correlated error. wrong wrong wrong ↓ 3-0 consensus Nothing about that creates truth. So multi-agent consensus should be interpreted as one signal. External verification remains stronger. agent agreement ↓ interesting evidence external postcondition ↓ stronger evidence If the task has an authoritative verifier, use it. Do not replace it with democracy among models. 12. Disagreement Should Be Typed “The agents disagree” is not a useful diagnosis. They may disagree about very different things. For example: EVIDENCE_CONFLICT STATE_CONFLICT INTERPRETATION_CONFLICT PLAN_CONFLICT AUTHORITY_CONFLICT VERIFICATION_CONFLICT RESOURCE_CONFLICT INTENT_CONFLICT Each requires different handling. If two agents disagree because they observed different repository commits, the answer is not more debate. It is state reconciliation. If they disagree because one used stale policy, the answer is policy refresh. If they disagree because two plausible interpretations remain, then adversarial reasoning may help. This is exactly the same lesson as typed uncertainty from Step 17. Do not spend debate on disagreements that authoritative state can resolve directly. 13. Build a Conflict Resolver, Not a Debate Loop A simple conflict object might look like: @dataclass(frozen=True) class CoordinationConflict: conflict_id: str conflict_type: str participants: tuple[str, ...] artifact_refs: tuple[str, ...] state_refs: tuple[str, ...] blocking: bool Resolution policy might be: STATE_CONFLICT -> fetch authoritative current state EVIDENCE_CONFLICT -> inspect source provenance / gather stronger evidence AUTHORITY_CONFLICT -> consult control plane OWNERSHIP_CONFLICT -> compare lease/epoch PLAN_CONFLICT -> rank candidates under common verifier VERIFICATION_CONFLICT -> strengthen verifier / independent adjudication Notice how little of this requires agents arguing with each other. That is intentional. 14. Prevent Duplicate Work Before It Starts Multi-agent systems often waste enormous compute because several workers independently rediscover the same thing. Sometimes duplication is useful. Independent candidates can reduce variance. But duplication should be deliberate. From Step 19: intentional duplicate computation ≠ duplicate side effect A work registry can identify existing assignments. @dataclass(frozen=True) class WorkKey: intent_id: str intent_version: int task_family: str target_state_hash: str Before launching a new worker: same work already active? ├─ yes -> join / await / intentionally fork └─ no -> claim assignment This can support single-flight execution for deterministic expensive work. But do not automatically coalesce independent verification or intentionally diverse search. The system should know why duplication exists. 15. Parallelism Requires Isolation Suppose two coding agents explore different fixes. They should not share one mutable checkout. Use isolated workspaces: base repository snapshot │ ├── workspace A ├── workspace B └── workspace C Each candidate preserves lineage. Then selection happens before any shared mutation. This is the same speculative-execution principle from Step 19. The multi-agent version is: Parallel agents may share immutable evidence. They should not casually share mutable working state. Otherwise one agent’s partial work contaminates another’s evidence. 16. Use Merge Semantics Appropriate to the Artifact Agent work often converges. But “merge the outputs” can mean very different things. For text research: union source sets; resolve duplicate claims; preserve conflicting evidence. For code: structural merge; conflict detection; tests; semantic verification. For plans: preserve commitments; compare dependencies; recompute critical path. For operational actions: merging may be impossible. The system should not ask a model to magically combine incompatible side effects. Artifact type determines merge semantics. 17. Model Delegation as a Contract Delegation should create an explicit contract between coordinator and worker. A useful contract has at least: input identity scope authority ceiling expected artifact acceptance criteria deadline budget failure policy handoff policy For example: @dataclass(frozen=True) class DelegationContract: assignment_id: str input_refs: tuple[str, ...] allowed_outputs: tuple[str, ...] authority_class: str budget_id: str acceptance_contract: str failure_policy: str handoff_policy: str The worker may choose how to reason inside that boundary. But the boundary itself remains platform state. This is analogous to function contracts in ordinary software. A good interface reduces the amount of coordination required. 18. Separate Coordinator From Worker One useful architecture is: Coordinator │ ┌───────────┼───────────┐ │ │ │ Worker A Worker B Worker C │ │ │ └───────────┼───────────┘ │ Shared evidence │ Verifier But be careful. The coordinator should not automatically become an omnipotent model. Its responsibilities may be largely deterministic: assign work; enforce ownership; track budgets; observe completion; invoke resolution policy; advance workflow state. A model may assist with task decomposition or routing. But the coordinator’s control authority should remain constrained. This prevents one central model from becoming both planner and unreviewed root authority. 19. Decentralization Does Not Remove the Need for Authority A common temptation is: If there is no central coordinator, the agents can negotiate among themselves. That can work for some low-risk collaborative search. But consequential shared resources still need authority semantics. Two agents cannot negotiate away a database uniqueness constraint. Two deployment agents cannot vote themselves into owning the same fencing token. Two workers cannot mutually declare a stale approval valid. Distributed coordination still needs authoritative state somewhere. That authority may itself be distributed. But it must be enforceable. 20. Use Leases for Liveness, Fencing for Correctness This distinction is worth repeating. A lease answers: Who appears to own this work right now? A fencing token answers: Which owner generation is still permitted to mutate the resource? Suppose Agent A’s lease expires during a network partition. Agent B receives epoch 19. Agent A later wakes up holding epoch 18. If the mutation gateway checks only local lease status, A may still cause damage. If the external mutation requires current fencing epoch: A epoch 18 -> reject B epoch 19 -> permit if all other gates pass This makes failover safe even when stale workers remain alive. 21. Deadlocks Are Real Once agents can own commitments and wait on one another, deadlocks appear. Example: Agent A owns commitment C1 waiting for artifact B Agent B owns commitment C2 waiting for approval from A Neither progresses. A coordination runtime should maintain a wait-for graph. A -> B B -> A Cycle detected. The response might be: break delegation; escalate; transfer ownership; revoke one commitment; choose alternative plan. Do not rely on conversation timeouts alone. The workers may both be healthy. The system is still deadlocked. 22. Livelock Is Different Agents can also repeatedly react to each other without making progress. A revises plan B critiques revision A revises critique B requests new evidence A changes plan B reopens prior concern ... No worker is blocked. But the goal does not advance. Measure progress using external state or workflow transitions. Possible livelock signals: high message count high replanning count low verified state change no commitment resolution repeated conflict signature Then stop the loop. Escalate, gather stronger evidence, simplify the architecture or choose a deterministic resolution rule. 23. Prevent Delegation Cycles Delegation graphs can become pathological too. A delegates to B B delegates to C C delegates to A Or: A -> B -> C -> D -> E -> ... until the original task is buried under orchestration. Useful safeguards include: maximum delegation depth; explicit parent assignment; cycle detection; inherited budget ceilings; inherited authority ceilings; delegation-cost accounting. Delegation should not create resources or authority from nothing. A child assignment consumes the parent’s budget envelope. 24. Preserve Budget Ownership Across Agents Without a shared ledger, every agent can believe it has the full task budget. That creates multiplicative spend. If the run has: $5 model budget 200 tool calls 20 minutes wall-clock target and four agents each behave as though those limits are local, the platform can consume four times the intended resources. Budgets should therefore live at the workflow/run level. Workers receive allocations. run budget │ ├── Agent A allocation ├── Agent B allocation ├── Agent C allocation └── protected verification reserve Agents may request more. They should not silently mint more. 25. Preserve Verification Capacity This follows directly from Step 16 and Step 21. Multi-agent search can consume all resources if allowed. Then the platform reaches a candidate but cannot afford to verify it. That is backwards. A multi-agent coordinator should reserve capacity for: final verification; reconciliation; required cleanup; authoritative state refresh; critical human escalation. Optional debate loses priority before required verification does. 26. Multi-Agent Search Should Still Earn Its Cost Multiple agents are expensive. The correct baseline is not: Did several agents produce a good result? The baseline is: single strong model simple generate+verify beam search MCTS multi-agent specialists multi-agent debate under comparable budgets. Measure: verified success; false success; UNKNOWN rate; latency; cost per verified success; marginal gain from additional workers. If three agents cost 4x as much and produce no measurable improvement, remove them. Multi-agent architecture is not a prestige feature. 27. Specialization Must Be Demonstrated An agent named SecurityExpert is not automatically a security expert. A role label is not competence evidence. The Step 30 competence-envelope rules still apply. specialist label ≠ demonstrated specialist competence Routing should therefore depend on evidence: Task family X ↓ Agent profile A: VALIDATED Agent profile B: LIMITED Agent profile C: UNVALIDATED That gives us evidence-backed heterogeneous teams rather than theatrical personas. 28. Joint Competence Is Its Own Property Suppose: Agent A succeeds 95% on task A Agent B succeeds 95% on task B Verifier C succeeds 98% on verification C That does not prove the composed workflow is reliable. Interfaces may fail. Artifacts may be misinterpreted. State identities may mismatch. Handoff may lose information. Authority may be mis-scoped. The composed system needs its own competence evidence. component competence ≠ system competence Benchmark the actual coordination path. 29. Independence Must Be Real Suppose two “independent” agents use: the same base model; the same provider; the same retrieval source; the same prompt template; the same verifier. They are not independent in many relevant failure dimensions. Track failure domains. model_family provider region retrieval_index prompt_family verifier_family training lineage if known This matters for: adversarial review; fallback; redundancy; reliability analysis; correlated false PASS. Two wrappers around the same failure domain do not create genuine redundancy. 30. Multi-Agent Memory Needs Provenance Shared memory can be useful. But one agent’s old conclusion should not become timeless truth for another. Memory entries should preserve: source agent source evidence state identity intent identity creation time verification status expiry/freshness semantics From Steps 36 and 41: Memory is evidence about prior state, not current authority. This remains true in multi-agent systems. A second agent reading a memory entry does not refresh it merely by restating it. 31. Handoff Must Preserve Meaning Step 35 already gave us portable execution state. Multi-agent handoff is the same problem plus ownership transfer. A safe handoff includes: assignment identity intent version state version artifact refs evidence refs budget remaining commitment refs ownership epoch authority envelope pending verification The receiving agent validates compatibility before activation. Then old ownership is fenced. Do not treat: “Here’s the conversation so far” as a complete handoff protocol. 32. Messages Need Delivery Semantics If agents communicate through queues, expect duplicates and delay. A message may be: delivered twice; delivered after cancellation; delivered after supersession; delivered after ownership transfer; delivered out of order. Therefore structured coordination messages should have identities. @dataclass(frozen=True) class CoordinationMessage: message_id: str intent_ref: IntentRef assignment_id: str sender_id: str recipient_id: str message_type: str payload_ref: str created_at: datetime Consumers should be idempotent where possible. And every consequential reaction must still check current control-plane state. A delayed message is not a time machine. 33. Events Should Wake Work, Not Authorize It This is the durable-workflow rule from Step 39. Suppose Agent B finishes research and emits: RESEARCH_COMPLETE Agent A wakes up. That event tells A something happened. It does not prove: the current intent still wants the work; the evidence is still fresh; the artifact passed verification; A still owns the next step; current policy permits mutation. So: event ↓ wake workflow ↓ revalidate ↓ continue if permitted This prevents stale agent messages from resurrecting obsolete work. 34. Use Structured Status, Not Conversational Guessing A coordinator should not need to ask: Are you done yet? and interpret prose. Workers should expose structured status: PENDING RUNNING WAITING_EVIDENCE WAITING_AUTHORITY WAITING_DEPENDENCY VERIFYING COMPLETED FAILED CANCELLED RECONCILIATION_REQUIRED That gives the workflow engine durable semantics. The model may explain the status. The explanation is not the status itself. 35. Conflict Resolution Should Prefer Stronger Evidence When two agents disagree, do not automatically choose: majority vote; more confident answer; more verbose answer; higher-status role. Prefer evidence hierarchy. For example: authoritative external state > deterministic verifier > independent domain evidence > model judgement > model confidence This keeps disagreement resolution aligned with the evidence-first architecture of the whole series. 36. Allow NO_SAFE_COORDINATION_PATH The platform should be allowed to conclude that no safe coordination plan exists. Perhaps: ownership state is ambiguous; required verifier is unavailable; agents disagree on an irreversible action; state versions cannot be reconciled; authority transfer cannot be established; the only remaining worker is outside its competence envelope. The correct outcome may be: NO_SAFE_COORDINATION_PATH or: HUMAN_REQUIRED Do not force consensus because the architecture expects an answer. 37. Multi-Agent Coordination Is a Workflow Graph A mature multi-agent system looks less like a chat room and more like a typed workflow graph. Intent v14 │ Goal / Workflow │ ┌────────────┴────────────┐ │ │ Assignment A Assignment B repository scan migration check │ │ Agent A Agent B │ │ Evidence EA Evidence EB └────────────┬────────────┘ │ Candidate C │ Independent V │ Verification │ Mutation Gateway │ Side Effect Natural-language communication may exist inside this graph. But the graph defines operational truth. 38. Coordination State Should Be Queryable Operators should be able to ask: Who currently owns commitment C? Which workers are serving intent v14? Which assignments depend on stale state? Which verifier blocked deployment? Which agent last held mutation authority? What work is duplicated? Which assignments are waiting on Agent B? Where is the deadlock cycle? What evidence supports candidate C? If those questions require reading hundreds of chat messages, the system is not operationally observable enough. The answer should come from structured state. 39. Add Coordination Trajectory Events Step 13 gave us trajectory observability. Multi-agent systems need coordination-specific events. For example: ASSIGNMENT_CREATED ASSIGNMENT_ACCEPTED WORK_DELEGATED OWNERSHIP_ACQUIRED OWNERSHIP_TRANSFER_PREPARED OWNERSHIP_TRANSFERRED EVIDENCE_PUBLISHED CONFLICT_DETECTED CONFLICT_RESOLVED DEADLOCK_DETECTED CANCELLATION_PROPAGATED STALE_MESSAGE_REJECTED DUPLICATE_WORK_COALESCED VERIFICATION_REQUESTED VERIFICATION_COMPLETED COMMITMENT_RELEASED These events make multi-agent failures reconstructable. Without them, incident analysis degenerates into transcript archaeology. 40. Metrics Should Measure Coordination Quality Useful metrics include: duplicate_work_rate coordination_overhead_ratio assignment_handoff_count ownership_conflict_rate stale_message_rate conflict_resolution_latency deadlock_rate livelock_rate verification_independence_rate commitment_orphan_rate handoff_failure_rate coordination_cost_per_verified_success Also track: single-agent baseline Otherwise a complicated multi-agent architecture may look impressive while performing worse than one well-instrumented worker. 41. Coordination Overhead Is Real Work Every additional agent introduces costs: context construction; messages; synchronization; artifact serialization; state refresh; handoff; conflict resolution; verification; scheduling; observability. That overhead should be measured. A useful ratio is: coordination overhead --------------------- verified useful work If coordination cost dominates useful work, the team is too complicated. The answer may be fewer agents. 42. Use Deterministic Coordination Where Possible Not every routing or ownership decision needs an LLM. Examples that should usually be deterministic: lease validity fencing epoch intent identity assignment ownership budget remaining commitment state authority scope message deduplication workflow state hard dependency readiness A model may help with: decomposing ambiguous work; judging semantic overlap; proposing a specialist; interpreting conflicting evidence; suggesting alternative plans. But exact control facts should remain exact. 43. Example: Coding Agents Suppose we want several coding agents to repair a repository. A naive architecture: three agents share repo three agents edit files one critic comments merge everything A stronger architecture: Repository snapshot R17 │ ├── Assignment A: diagnose failing test │ └── Agent A -> Evidence EA │ ├── Assignment B: inspect call graph │ └── Agent B -> Evidence EB │ └── Assignment C: search historical fixes └── Agent C -> Evidence EC EA + EB + EC ↓ planner proposes repair candidates ↓ isolated workspaces ↓ independent test/verifier pipeline ↓ selected candidate ↓ merge authority gate No agent needs direct production repository mutation authority during exploration. This sharply reduces coordination risk. 44. Example: Research Agents Research is different. Multiple agents may deliberately search different evidence domains. Agent A -> academic papers Agent B -> official statistics Agent C -> regulatory sources Agent D -> adversarial counter-evidence Each publishes claim/evidence artifacts. A synthesis agent receives: claim IDs source provenance publication dates conflicts freshness source authority not merely four prose essays. If sources conflict, the conflict remains visible. The synthesizer should not erase disagreement to produce cleaner prose. 45. Example: Browser Agents Browser workflows are especially dangerous because multiple agents may interact with the same mutable session. Do not casually allow: Agent A edits cart Agent B changes address Agent C applies coupon Agent D clicks purchase on one live browser state. Better: read-only discovery agents ↓ structured candidate purchase ↓ single owned mutation session ↓ revalidation ↓ approval if required ↓ commit ↓ postcondition verification Parallelize information gathering. Serialize consequential shared-session mutation unless the application supplies stronger transaction semantics. 46. Example: DevOps Agents Imagine: Agent A -> logs Agent B -> metrics Agent C -> traces Agent D -> deployment history They can gather evidence in parallel. Then a coordinator builds an incident hypothesis set. But only one recovery workflow should own the consequential production mutation commitment unless an explicit plan divides resources safely. The recovery agent receives narrowly scoped credentials. A separate verifier checks postconditions. This gives us specialization without turning every diagnostic agent into a production operator. 47. Example: Mixture of Agents Earlier in this series we covered Mixture-of-Agents architectures. Now we can sharpen the model. There are at least three different things people call “Mixture of Agents”: 1. multiple candidate generators 2. specialist routing 3. cooperative operational workers They need different infrastructure. Candidate generation may require only artifact identity and ranking. Specialist routing requires competence evidence. Operational cooperation requires everything in this chapter: ownership; authority; commitments; durable workflows; state consistency; trust boundaries; transaction recovery. Do not import operational complexity into a simple candidate-generation problem unless you need it. 48. Failure Mode: Everyone Owns Everything This produces: responsibility duplication mutation races unclear accountability orphan commitments Fix: explicit assignment ownership; commitment ownership; fencing; narrow delegation. 49. Failure Mode: Nobody Owns Cleanup A workflow fails. Every worker considers its task complete. But one reservation, temporary branch or external job remains active. Fix: commitments survive task completion; cleanup/reconciliation gets explicit ownership; workflow cannot terminally complete with unresolved required commitments. 50. Failure Mode: Persuasive Consensus Several agents repeat the same plausible explanation. The system interprets repetition as truth. Fix: external verification; source provenance; independence analysis; correlated-error awareness. 51. Failure Mode: Hidden Shared State Agents communicate through a common scratchpad. One agent overwrites a fact another still assumes is current. Fix: immutable artifact identity; explicit state versions; derived facts tied to provenance; controlled mutable registries. 52. Failure Mode: Delegation Launders Authority A privileged parent delegates to an untrusted child. The child inherits broad credentials. Fix: authority narrowing; credential broker; capability-scoped grants; deterministic mutation gateway. 53. Failure Mode: Endless Debate Agents keep producing critiques because no explicit resolution policy exists. Fix: typed conflicts; evidence hierarchy; stopping conditions; escalation; budget ceilings. 54. Failure Mode: Retry Creates a Second Agent Team A workflow times out and retries by launching a new team while the original workers remain alive. Fix: stable assignment IDs; work registry; ownership leases; fencing; idempotent event handling. 55. Failure Mode: Verification Is Inside the Same Coalition All agents optimize toward getting the workflow accepted. The verifier becomes socially or informationally aligned with the candidate team. Fix: separate verifier role; restricted information flow; independent acceptance criteria; verifier release governance. 56. Failure Mode: Shared Model Means Fake Diversity Five agents use the same model and prompt family. The system calls this five independent opinions. Fix: record failure-domain identity; measure disagreement correlation; compare against a single model with extra samples. 57. Failure Mode: Coordination State Lives in Prompts Prompts contain statements such as: You are agent 3. Agent 2 owns the deployment. The user approved rollback. Those statements may be stale or attacker-controlled. Fix: inject authoritative structured control state; keep control facts outside free-form content; validate again at mutation boundaries. 58. Failure Mode: Agent Hierarchy Becomes a Power Hierarchy A manager agent delegates to worker agents and is therefore assumed to be more correct. That is unjustified. Organizational role does not imply epistemic superiority. The coordinator may have orchestration authority while having less domain competence than a specialist. Separate: coordination authority from domain competence from verification authority 59. Coordination Needs Failure Injection Do not test multi-agent systems only when everyone behaves perfectly. Inject: worker crash after producing artifact worker crash after external commit stale ownership epoch duplicate completion event delayed cancellation conflicting state snapshot malicious agent message forged verification claim agent outside competence envelope deadlock cycle livelock loop provider outage budget exhaustion human approval expiry commitment-owner disappearance handoff corruption Then verify platform invariants. Examples: no duplicate consequential effect no stale owner mutation no authority expansion through delegation no workflow completion with unresolved required commitments no verification claim accepted without evidence no cancelled intent resurrected by delayed message 60. A Minimal Coordination Runtime A simple deterministic core might look like this: from dataclasses import dataclass from enum import Enum class AssignmentStatus(str, Enum): PENDING = "pending" RUNNING = "running" WAITING = "waiting" VERIFYING = "verifying" COMPLETED = "completed" FAILED = "failed" CANCELLED = "cancelled" @dataclass(frozen=True) class Assignment: assignment_id: str intent_id: str intent_version: int owner_id: str ownership_epoch: int status: AssignmentStatus authority_scope: tuple[str, ...] artifact_refs: tuple[str, ...] class CoordinationGate: def may_mutate( self, *, assignment: Assignment, current_intent_version: int, current_owner_id: str, current_epoch: int, requested_capability: str, verification_passed: bool, ) -> bool: if assignment.intent_version != current_intent_version: return False if assignment.owner_id != current_owner_id: return False if assignment.ownership_epoch != current_epoch: return False if requested_capability not in assignment.authority_scope: return False if not verification_passed: return False return True This is deliberately boring. That is good. The model can be sophisticated inside the assignment. The coordination boundary should remain understandable. 61. Do You Actually Need Multiple Agents? Before adopting this architecture, ask: Does the task genuinely benefit from parallel independent search? Do we have heterogeneous specialist competence? Can work be decomposed cleanly? Can results be independently verified? Does parallelism reduce critical-path latency? Are failures sufficiently independent? Does the benefit exceed coordination overhead? If not, use one agent. Or deterministic software. Or one model plus a verifier. The simplest system that reaches the reliability target is usually the better architecture. 62. The Deeper Pattern Early agent systems often imagined multi-agent intelligence as a society of minds. One agent plans. One criticizes. One researches. One manages. That metaphor can be useful. But production systems need a more precise model. agents = disposable reasoning/execution workers shared control state = operational truth contracts = delegation semantics leases/fencing = ownership correctness provenance = evidence lineage verifiers = acceptance evidence workflow = durable coordination The interesting property is not that several models can talk. It is that several uncertain workers can contribute to one objective without gaining ambiguous ownership or uncontrolled authority. 63. The Architecture So Far We can now see the larger platform clearly. INTENT │ GOALS / WORKFLOW │ COMMITMENT GRAPH │ ASSIGNMENT / OWNERSHIP │ ┌────────────┼────────────┐ │ │ │ Worker A Worker B Worker C │ │ │ evidence candidate evidence └────────────┼────────────┘ │ CONFLICT POLICY │ VERIFICATION │ AUTHORITY GATE │ SIDE EFFECT │ POSTCONDITION CHECK │ PROVENANCE Around that: leases fencing budgets placement security replay SLOs cancellation reconciliation behavioral releases This is no longer “a bunch of agents.” It is a controlled autonomy platform with multiple workers. 64. What Comes Next We now have almost all of the execution-plane machinery. The next question is architectural rather than tactical: Who controls all of these mechanisms? Intent versioning exists. Competence envelopes exist. Authority policy exists. Placement exists. Budgets exist. Reliability policy exists. Capability portfolios exist. Release policy exists. Security policy exists. Multi-agent ownership exists. At this point we should stop treating those as unrelated utilities. They form a control plane. That is the next stage. Conclusion Multi-agent systems do not become reliable because the agents communicate more. They become reliable when cooperation has explicit semantics. shared intent + scoped assignments + explicit ownership + commitment semantics + narrow authority + durable workflow state + evidence provenance + typed conflict resolution + independent verification + fencing/idempotency + observable coordination The central principle is worth repeating: Multiple agents should coordinate through explicit contracts and shared state, not by merely chatting until they agree. And the broader lesson is the same one that has emerged throughout this series: Advanced agent capability is useful only when the surrounding system can make its uncertainty, authority, evidence, ownership and effects explicit. In the next chapter we will pull those mechanisms together into one architectural object: the agent control plane.