# Advanced Agents From First Principles 10: Are You Combining Every Agent Technique Into One Monster? Build a Mixture-of-Agents Runtime
# Book: Advanced Agents From First Principles
# Chapter URL: https://programmer.ie/books/advanced-agents-from-first-principles/10-chapter/
# Prose fingerprint (sha256, normalized): d218586f6d3c421f31165327021669025f791156c9f4c95135680366544b7ad7
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

You have a working agent.
Then you add retrieval.
Then memory.
Then Best-of-N.
Then critique and revision.
Then Tree of Thoughts.
Then MCTS.
Then specialist models.
Then adversarial review.
Then a planner, executor, critic and verifier.
Then a stronger model for hard cases.
Eventually the architecture starts to look like this:
request ↓ planner ↓ retrieval ↓ reasoning ↓ Best-of-N ↓ critic ↓ Tree of Thoughts ↓ MCTS ↓ frontier model ↓ second critic ↓ verifier Every mechanism may have been individually reasonable.
The combined system can still be terrible.
It is slow.
It is expensive.
It is difficult to debug.
It is difficult to know which component actually helped.
And most importantly:
every request pays for every capability whether it needs it or not.
The answer is not to build an even larger agent.
The answer is to stop treating advanced techniques as permanent layers.
Treat them as alternative strategies.
Then route between those strategies.
That gives us an agent-level architecture that is closer to a Mixture of Agents:
┌→ direct model ├→ deterministic workflow ├→ retrieval + answer request → strategy router ├→ Best-of-N ├→ critique + revise ├→ Tree of Thoughts ├→ MCTS ├→ specialist runtime ├→ adversarial review └→ human escalation ↓ verification The important idea is not the name.
The important idea is this:
Do not combine every reasoning mechanism into one agent. Route the task to the smallest strategy that can solve it reliably.
This is the architectural synthesis of the advanced-agent techniques we have built so far.
The Problem: Architecture Accretion Agent systems often become complicated one feature at a time.
A production failure appears.
Someone adds another mechanism.
failure ↓ new mechanism ↓ new failure ↓ new mechanism The individual decisions can all make sense.
The failure is that the mechanisms become stacked rather than selected.
Suppose a simple repository question can be answered by reading one file.
A badly accreted system might still perform:
planning, retrieval, multiple reasoning samples, critic review, plan revision, specialist routing, final verification. The task may cost ten times more without becoming meaningfully more reliable.
The system has confused:
capabilities available to the runtime with:
capabilities that must execute on every request Those are completely different things.
From Mixture of Experts to Mixture of Agents In the previous routing post we built an agent-level Mixture of Experts.
That architecture routed work between different capabilities:
task ↓ router ├→ local model ├→ frontier model ├→ code specialist ├→ retriever ├→ deterministic tool └→ verifier A Mixture-of-Agents runtime moves the routing boundary upward.
Instead of routing only between experts, it can route between entire control strategies.
For example:
Strategy: DIRECT model → answer → verify or:
Strategy: RETRIEVAL retrieve → answer → verify or:
Strategy: SEARCH branch → evaluate → prune → continue → verify or:
Strategy: MCTS select → expand → evaluate → backpropagate → verify or:
Strategy: REVIEW proposal → adversarial critics → evidence checks → adjudicate These are not merely different prompts.
They are different runtime policies for spending computation.
The Core Abstraction: A Strategy A useful strategy interface is surprisingly small.
from dataclasses import dataclass from typing import Protocol, Any @dataclass class StrategyResult: output: Any status: str evidence: list[dict] cost: float latency_ms: int calls: int class AgentStrategy(Protocol): name: str def supports(self, state: dict) -> bool: ... def estimate_cost(self, state: dict) -> float: ... def run(self, state: dict) -> StrategyResult: ... The strategy does not need to know about every other strategy.
It only needs a contract:
input state ↓ strategy ↓ structured result + evidence + resource usage That gives the runtime something it can compare.
A Strategy Registry Now we can register available strategies.
class StrategyRegistry: def __init__(self): self._strategies = {} def register(self, strategy: AgentStrategy): self._strategies[strategy.name] = strategy def available(self, state: dict): return [ strategy for strategy in self._strategies.values() if strategy.supports(state) ] A real registry might contain:
DIRECT DETERMINISTIC_WORKFLOW RETRIEVAL BEST_OF_N CRITIQUE_REVISE TREE_OF_THOUGHTS MCTS SPECIALIST_ROUTING PLANNER_EXECUTOR_CRITIC ADVERSARIAL_REVIEW HUMAN_ESCALATION The important difference is architectural:
these mechanisms are alternatives available to the controller, not mandatory stages.
What Should the Router Look At? A strategy router needs state.
Not just the user prompt.
Useful routing signals include:
task type risk current verification status available evidence uncertainty previous failures remaining budget side-effect severity latency target required precision search-space size known deterministic path Represent them explicitly.
@dataclass class StrategyContext: task_type: str risk: float uncertainty: float previous_failures: int verification_status: str remaining_budget: float deterministic_path_known: bool irreversible_action: bool Then routing becomes a decision over system state.
Start With a Deterministic Router Do not immediately train another model to route your agents.
Start with rules.
def choose_strategy(ctx: StrategyContext) -> str: if ctx.deterministic_path_known: return "DETERMINISTIC_WORKFLOW" if ctx.risk < 0.2 and ctx.uncertainty < 0.2: return "DIRECT" if ctx.task_type == "fact_lookup": return "RETRIEVAL" if ctx.previous_failures == 0 and ctx.uncertainty < 0.5: return "CRITIQUE_REVISE" if ctx.previous_failures >= 1 and ctx.uncertainty < 0.8: return "TREE_OF_THOUGHTS" if ctx.remaining_budget > 5.0: return "MCTS" return "HUMAN_ESCALATION" This is intentionally crude.
That is useful.
You can inspect it.
You can measure it.
You can discover where it fails.
Only then do you have evidence that a learned router may be worth building.
Strategy Routing Is Not Model Routing These are different decisions.
Model routing task ↓ local model / frontier model / specialist model Strategy routing task state ↓ direct / retrieval / revision / search / MCTS / review A strategy can itself contain model routing.
For example:
TREE_OF_THOUGHTS strategy ↓ local model generates branches ↓ learned scorer ranks branches ↓ frontier model handles difficult branch ↓ external verifier checks result This produces a hierarchy:
strategy router ↓ strategy ↓ expert/model/tool router Do not collapse those layers unless the simpler representation actually works better.
The Simplest Strategy Should Be a First-Class Expert Advanced systems often make a subtle mistake.
They include many sophisticated routes but forget to include:
just answer the question That should be a real strategy.
class DirectStrategy: name = "DIRECT" def supports(self, state): return True def estimate_cost(self, state): return 0.01 def run(self, state): output = call_model(state["prompt"]) return StrategyResult( output=output, status="PROPOSED", evidence=[], cost=0.01, latency_ms=300, calls=1, ) Why?
Because the direct strategy is the baseline every advanced strategy must beat.
If Tree of Thoughts takes twenty calls and improves verified success from 96% to 96.2%, that may be a terrible trade.
Strategy Selection Needs Verification A router can be perfectly confident and still choose the wrong strategy.
So the runtime should not assume:
route selected = task solved Instead:
route selected ↓ strategy runs ↓ external verification ↓ PASS / FAIL / UNKNOWN The verifier produces the feedback that allows strategy selection to become adaptive.
Failure Should Change Strategy, Not Just Repeat It A common agent anti-pattern is:
strategy fails ↓ run same strategy again ↓ run same strategy again That is retry, not adaptation.
A better controller asks:
what failure occurred? Then changes strategy.
For example:
DIRECT failed because evidence missing ↓ RETRIEVAL RETRIEVAL failed because sources conflict ↓ ADVERSARIAL_REVIEW DIRECT produced several plausible implementations ↓ BEST_OF_N Best-of-N candidates all fail tests differently ↓ TREE_OF_THOUGHTS Tree search has delayed payoff and weak early scores ↓ MCTS The escalation is failure-specific.
A Runtime Controller A simple controller might look like this:
class MixtureOfAgentsRuntime: def __init__(self, registry, router, verifier): self.registry = registry self.router = router self.verifier = verifier def run(self, state: dict): history = [] while True: strategy = self.router.choose(state, history) result = strategy.run(state) verification = self.verifier.verify(state, result) history.append({ "strategy": strategy.name, "result": result, "verification": verification, }) if verification.status == "PASS": return result, history if verification.status == "FAIL": state = update_state_from_failure( state, verification, ) if verification.status == "UNKNOWN": state = request_more_evidence( state, verification, ) if budget_exhausted(state, history): return None, history Notice what is missing.
There is no giant universal agent prompt.
The runtime controls composition.
Dynamic Composition Sometimes one strategy is not enough.
The router may compose strategies sequentially.
For example:
RETRIEVAL ↓ DIRECT ↓ VERIFY or:
TREE_OF_THOUGHTS ↓ BEST_OF_N on final branches ↓ VERIFY or:
PLANNER_EXECUTOR ↓ ADVERSARIAL_REVIEW ↓ VERIFY The danger is that dynamic composition can recreate the original monster architecture.
So composition needs the same rule:
Every added stage must be triggered by a measured failure or uncertainty.
Do not compose mechanisms merely because they are available.
The Monster-Agent Failure Mode Suppose every request always runs:
planner + retrieval + self-consistency + Tree of Thoughts + MCTS + two critics + frontier verifier That architecture has almost no useful adaptation.
It is simply a very expensive fixed pipeline.
A mixture architecture should instead look like:
Easy request → DIRECT Known deterministic task → WORKFLOW Knowledge gap → RETRIEVAL Output variance → SELF_CONSISTENCY Premature reasoning commitment → TREE_OF_THOUGHTS Delayed-payoff search → MCTS Specialist capability needed → EXPERT_ROUTING High-risk unresolved claims → ADVERSARIAL_REVIEW Different tasks should produce visibly different trajectories.
If they do not, your router is probably decorative.
Measure Strategy Utilization Track how often each strategy is selected.
DIRECT 54% RETRIEVAL 18% CRITIQUE_REVISE 11% TREE_OF_THOUGHTS 7% SPECIALIST_ROUTING 5% MCTS 2% ADVERSARIAL_REVIEW 2% HUMAN_ESCALATION 1% The target is not equal utilization.
Equal utilization may actually be suspicious.
If MCTS is only justified for 2% of tasks, 2% may be exactly right.
The question is:
did each strategy earn the tasks it received? Strategy Confusion Matrix If you have labeled evaluation tasks, build a confusion matrix.
selected D R T M A actual DIRECT 81 4 3 0 2 actual RETR. 5 44 2 0 1 actual TOT 3 1 21 4 2 actual MCTS 0 0 5 12 1 This reveals systematic routing mistakes.
For example:
MCTS tasks → Tree of Thoughts may indicate the router cannot detect delayed reward.
Or:
DIRECT tasks → MCTS may indicate severe over-escalation.
Oracle Strategy Selection A powerful offline diagnostic is oracle strategy selection.
For each benchmark task, run every eligible strategy.
Record which ones produce a verified success.
Task 17 DIRECT FAIL RETRIEVAL PASS BEST_OF_N PASS TREE_OF_THOUGHTS PASS MCTS PASS The cheapest successful strategy is RETRIEVAL.
That gives us an oracle target:
oracle strategy = RETRIEVAL Now compare the router.
If it chose MCTS, the task succeeded but the router wasted compute.
If it chose DIRECT and stopped after failure, the router missed an available successful strategy.
Strategy Regret Define strategy regret as the difference between the selected strategy and the best available strategy under the objective.
The objective might include:
verified success cost latency risk For example:
utility = ( verified_success * 100 - cost * 4 - latency_seconds * 0.2 ) Then:
strategy regret = oracle utility - selected utility This is much more useful than raw routing accuracy.
A router that selects a slightly different but equally successful cheap strategy may be perfectly acceptable.
Success Per Unit Compute The main production metric should not be:
most sophisticated strategy selected It should be something closer to:
verified successful tasks ------------------------- compute / cost / latency For example:
Strategy Verified success Avg cost Avg latency Direct 82% $0.01 0.5s Retrieval 91% $0.02 0.8s Best-of-N 94% $0.06 2.2s ToT 95% $0.18 6.5s MCTS 96% $0.70 21s The conclusion is not:
MCTS wins The conclusion may be:
MCTS belongs only on the small set of tasks where cheaper strategies fail. That is exactly what the router is for.
Latency Budgets Matter A strategy may be technically superior and operationally unusable.
For an interactive coding assistant:
500 ms and:
30 seconds are not interchangeable.
So routing constraints should include latency.
@dataclass class StrategyBudget: max_cost: float max_latency_ms: int max_calls: int max_search_nodes: int A strategy that cannot fit inside the remaining budget should not be eligible.
Risk Changes Routing Difficulty and risk are different dimensions.
A simple action can be high risk.
Example:
delete production database The reasoning may be trivial.
The verification and authorization requirements are not.
So strategy selection should consider:
difficulty uncertainty risk separately.
A low-difficulty/high-risk task might route to:
deterministic workflow + strong authorization + independent verifier rather than to a complex reasoning strategy.
Authorization Is Not a Strategy Choice The router may choose a powerful executor.
That must not grant additional permissions.
router chooses deployment expert does not imply:
deployment expert may deploy to production Authorization remains an independent runtime boundary.
strategy selection ↓ capability request ↓ authorization check ↓ execution Never let a model route itself into broader authority.
Coding Agents A coding agent is an excellent example of strategy routing.
Direct Use for:
explain this function rename this local variable write a small pure helper Retrieval Use when repository knowledge is missing:
where is authentication configured? which module owns migrations? Deterministic workflow Use for known mechanical changes:
format lint run tests update generated files Best-of-N Use when several candidate implementations are cheap to generate and test.
Tree of Thoughts Use when multiple architectural directions should remain alive.
MCTS Use only when early implementation quality is a weak predictor of eventual test/integration success and search statistics can be reused.
Adversarial review Use for security, regression risk, API compatibility and architecture-boundary checks.
The repository state remains external truth throughout.
Research Agents Research strategy routing might look like:
known factual lookup → retrieval single clear primary source → retrieval + extraction conflicting sources → adversarial evidence review multiple plausible hypotheses → Tree of Thoughts long multi-stage investigation → planner-executor high disagreement after evidence → specialist adjudication The point is not to make every research task a debate.
Most factual lookups should stay cheap.
Customer Support Support agents benefit enormously from not treating all tickets equally.
password reset → deterministic workflow order status → database/tool lookup policy question → retrieval ambiguous billing dispute → specialist route high-value account escalation → human A giant reasoning agent is usually worse than a well-routed support system.
Data and Analytics Agents Different strategies fit different analytical tasks.
simple arithmetic → deterministic calculator known SQL query pattern → deterministic query builder schema discovery → retrieval/catalog tools ambiguous metric definition → reasoning + evidence several competing causal explanations → hypothesis search high-impact conclusion → adversarial review + verification Again, not everything should become multi-agent reasoning.
Browser Automation Browser agents have especially strong reasons to route strategies carefully because actions can have side effects.
read page → browser tool extract table → deterministic parser navigate known flow → workflow unknown site structure → planning/search purchase / submit / delete → authorization + explicit verification Search strategies should operate over read-only or reversible actions where possible.
DevOps and Incident Response A DevOps mixture might contain:
log retriever metrics analyzer runbook workflow root-cause hypothesis search rollback planner specialist reviewer human incident commander Routing should use live evidence:
alerts logs metrics recent deployments service topology current incident state Not just a textual description of the incident.
Router Confidence Is Not Success Confidence These are separate signals.
router confidence = how sure are we this strategy is appropriate? success confidence = how much evidence supports task completion? A router might be 99% sure that MCTS is the right strategy.
The MCTS result can still fail verification.
Do not collapse the two.
Strategy Routing Can Learn The previous post introduced learning from verified trajectories.
Strategy routing is an obvious policy surface.
A trajectory can record:
task features selected strategy alternative strategies tested offline verification outcome cost latency failure class From this we can learn candidate routing rules.
For example:
When repository tests fail after two independent direct patches, Tree of Thoughts has 31% higher verified success than repeated revision. That is a candidate lesson.
It still needs evaluation before promotion.
Do not let one successful MCTS run rewrite the router.
Shadow Routing A safe way to improve the router is shadow evaluation.
Production uses:
router_v4 while a candidate router computes:
router_v5_shadow without controlling execution.
Compare:
selected strategy oracle strategy verified outcome estimated cost actual cost Only then promote the new router.
Strategy-Level Failure Attribution Once strategies are explicit, failures become easier to classify.
Routing failure A better strategy existed but was not selected.
Strategy failure The selected strategy was appropriate but executed poorly.
Expert failure The strategy selected the wrong model/tool/specialist internally.
Budget failure The correct strategy was selected but lacked enough resources.
Verification failure The strategy succeeded but the verifier rejected it, or failed but the verifier passed it.
Authorization failure A strategy attempted a capability it was not allowed to use.
Learning failure Past verified evidence should have changed routing policy but did not, or a bad rule was promoted.
This separation is extremely valuable in production.
Do Not Build a Meta-Agent That Merely Thinks About Agents A common temptation is:
user request ↓ meta-agent ↓ "which agent should I use?" That can work.
But if the meta-agent is simply another unconstrained LLM call, you may have moved the problem instead of solving it.
Start with observable routing features and deterministic rules.
Add model-based strategy selection only when semantic classification genuinely helps.
Then measure its routing regret.
A Better Escalation Ladder A useful production sequence is:
1. deterministic path if known 2. direct model 3. retrieval / tool evidence 4. revision or Best-of-N 5. specialist routing 6. Tree of Thoughts 7. adversarial review 8. MCTS 9. stronger model / larger budget 10. human escalation This is not a universal ordering.
The correct ordering depends on the task.
But the principle is useful:
cheap, testable mechanisms first expensive, speculative mechanisms later Benchmark the Runtime, Not Just the Strategies Individual strategy benchmarks are not enough.
The mixture itself needs evaluation.
Compare:
Baseline A frontier model for everything Baseline B fixed advanced stack Baseline C deterministic rule router Baseline D adaptive router with verification feedback Baseline E learned router trained on verified trajectories Measure:
verified success cost latency strategy regret unnecessary escalation rate missed escalation rate human escalation rate failure recovery rate That is the real test.
Important Metrics Verified task success The only success metric that ultimately matters.
Strategy utilization How often each strategy runs.
Strategy regret How much utility was lost relative to the best available strategy.
Unnecessary escalation rate advanced strategy selected when cheaper strategy would have verified successfully Missed escalation rate cheap strategy failed and advanced strategy would have succeeded but was not selected Recovery rate How often strategy switching turns a failed attempt into verified success.
Cost per verified success The real economic denominator.
Latency per verified success Critical for interactive systems.
Strategy-switch count Too many switches may indicate thrashing.
Router calibration Does route confidence correspond to actual route quality?
Strategy Thrashing Adaptive systems can oscillate.
DIRECT → RETRIEVAL → DIRECT → RETRIEVAL → DIRECT Or:
TREE_OF_THOUGHTS → MCTS → TREE_OF_THOUGHTS That is strategy thrashing.
Prevent it with:
strategy history failure fingerprints switch limits no-progress windows budget limits The router should know what has already been tried.
The Runtime Should Preserve Shared State Strategies should not communicate by repeatedly summarizing the entire task in prose.
Use shared structured state.
@dataclass class AgentState: goal: str constraints: list[str] observations: list[dict] evidence: list[dict] failures: list[dict] attempts: list[dict] budget_remaining: float Then:
DIRECT fails and:
TREE_OF_THOUGHTS starts without losing the evidence already collected.
This avoids handoff loss across strategies.
Do Not Let Strategies Rewrite the Goal The user goal should remain immutable unless the user changes it.
Strategies may propose interpretations.
They may not silently redefine success.
original goal ↓ immutable goal contract ↓ strategy-specific plan The verifier checks against the original goal contract.
Not against whatever interpretation made the selected strategy look successful.
The Final Architecture We can now describe a fairly complete advanced runtime.
┌─────────────────────┐ │ Goal Contract │ └──────────┬──────────┘ ↓ ┌─────────────────────┐ │ Structured State │ └──────────┬──────────┘ ↓ ┌─────────────────────┐ │ Strategy Router │ └──────────┬──────────┘ ↓ ┌────────────────────────────────────────────────┐ │ │ ↓ ↓ ↓ ↓ ↓ Direct Retrieval Search MCTS Review │ │ │ │ │ └────────────┴─────────────┴────────────┴───────┘ ↓ ┌─────────────────────┐ │ External Verification│ └──────────┬──────────┘ ↓ PASS / FAIL / UNKNOWN / ESCALATE ↓ ┌─────────────────────┐ │ Trajectory Evidence │ └──────────┬──────────┘ ↓ offline learning loop This is not one giant intelligent agent.
It is a runtime that coordinates several bounded computational strategies.
That distinction matters.
The Deeper Pattern Across this entire series, the advanced techniques can now be seen as different answers to one question:
Where should the next unit of computation go?
Chain of Thought:
into another reasoning step Self-Consistency:
into another sample Tree of Thoughts:
into another branch MCTS:
into the branch whose exploration/exploitation value is highest Mixture of Experts:
into the specialist with the right capability Planner–Executor–Critic:
into the role responsible for the current failure Adversarial Review:
into testing the most important unresolved claim Adaptive Agents:
into whichever mechanism the current evidence justifies Learning:
into improving future allocation policy Mixture of Agents:
into the entire reasoning strategy with the highest expected utility That is the architectural endpoint.
Do You Actually Need a Mixture of Agents? Probably not at first.
Start here:
model ↓ structured action ↓ tool ↓ verification Only add strategy routing when you can show that different task classes genuinely benefit from different computational policies.
The evidence should look something like:
Task class A DIRECT: 98% verified success MCTS: 98% verified success, 25x cost Task class B DIRECT: 52% TREE_OF_THOUGHTS: 81% MCTS: 84% Task class C DIRECT: 70% RETRIEVAL: 96% Now routing has a reason to exist.
Without that evidence, a Mixture-of-Agents architecture may simply be another abstraction layer.
Production Checklist Before shipping strategy routing, ask:
Are strategies explicit runtime units rather than vague prompt roles? Is the direct/simple path a first-class strategy? Are routing features observable? Can deterministic rules solve routing adequately? Is every strategy externally verified? Are cost and latency part of routing? Are risk and difficulty modeled separately? Is authorization independent from strategy choice? Can failures trigger a different strategy rather than blind retry? Do strategies share structured state? Is the original goal immutable? Can you calculate oracle strategy performance offline? Can you measure strategy regret? Can you detect unnecessary escalation? Can you detect missed escalation? Can you detect strategy thrashing? Are strategy changes versioned and reversible? Does the mixture beat a fixed frontier-model baseline? Does it beat a fixed advanced-stack baseline? If not, the architecture is probably not ready.
Final Rule A powerful agent platform should not ask:
How many advanced techniques can we combine? It should ask:
What is the cheapest computational strategy that can produce a verified result for this task state? That changes the architecture completely.
The runtime stops being a giant stack of intelligence-shaped components.
It becomes a policy for allocating computation under evidence, cost, latency and risk constraints.
And that is the real point of a Mixture of Agents.
Do not build one monster agent. Build a runtime that knows which kind of agentic computation to spend next — and make every escalation earn its cost with verified evidence.
The next step is to stop looking at mechanisms one at a time and evaluate the system as a whole: how do you benchmark an advanced agent architecture without letting demos, model judges or cherry-picked tasks fool you?