# Advanced Agents From First Principles 08: Is Your Agent Spending the Same Compute on Every Task? Build Adaptive Agents That Escalate Only When Needed
# Book: Advanced Agents From First Principles
# Chapter URL: https://programmer.ie/books/advanced-agents-from-first-principles/08-chapter/
# Prose fingerprint (sha256, normalized): bfe722357726130dfb57eee6f0421a9f52cfffff6e7ae901e04bd6af430090dc
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

A production agent should not spend the same amount of compute on every task.
Some requests are easy:
parse a known structure, retrieve a known record, make a deterministic transformation, call one reliable tool, run one test, answer from a high-confidence source. Others are not:
the task is ambiguous, multiple reasoning paths disagree, the cheap model fails verification, the search space is large, the action has expensive consequences, the evidence is incomplete, the current plan is not making progress. If every task goes through the full advanced-agent stack, the system becomes slow, expensive and fragile.
If every task stays on the cheapest path, difficult cases fail unnecessarily.
The useful architecture is therefore not:
cheap agent or expensive agent It is:
start cheap ↓ measure uncertainty / failure ↓ choose the smallest escalation ↓ verify again ↓ stop when evidence is sufficient This is adaptive compute allocation.
The core rule is:
Do not spend more inference compute because a task feels difficult. Spend more when a measurable signal says the current path is insufficient.
That distinction turns a pile of advanced techniques into a runtime policy.
The problem: fixed compute budgets are usually wrong Suppose every coding request always does this:
frontier model ↓ 5 reasoning samples ↓ Tree of Thoughts ↓ MCTS ↓ 3 critics ↓ final verifier You may get a sophisticated system.
You may also get a very expensive way to rename a variable.
The opposite design is also bad:
one cheap local model ↓ done That path may be excellent for repetitive repository tasks and inadequate for a subtle concurrency bug.
The right question is not:
Which architecture is best?
It is:
Which architecture should this task receive at this point in the trajectory?
That is a routing problem over compute policies.
Adaptive agents are control systems An adaptive agent has at least five pieces:
current state ↓ cheap policy ↓ result + evidence ↓ uncertainty / failure signals ↓ escalation controller ↓ next compute policy The controller may choose among:
no escalation more tool evidence more reasoning steps more samples Best-of-N specialist expert stronger model Tree of Thoughts MCTS adversarial review human escalation The important point is that these are actions available to the runtime.
They are not a fixed ladder that every request must climb.
Start with the cheapest path that could plausibly succeed A good default is:
exact deterministic path ↓ if unavailable cheap model / local model ↓ if insufficient stronger reasoning or more evidence ↓ if insufficient search / specialists / frontier model For example, a coding assistant might start with:
git status ↓ static inspection ↓ local model patch ↓ run tests If the tests pass and the required behavior is verified, stop.
There is no reason to invoke MCTS merely because MCTS exists.
If the tests fail, the runtime now has a concrete signal that escalation may be justified.
What should trigger escalation? The strongest escalation signals come from the environment.
Examples:
required test failed, schema validation failed, browser state did not change as expected, deployment health degraded, retrieved evidence conflicts, required source is missing, plan made no measurable progress, critic found a verified defect, verifier returned UNKNOWN, route confidence is low, multiple samples disagree materially, a tool action repeatedly fails, the current expert has low historical success on this task class. Weak signals include:
the model says the task is hard, the model says it is uncertain, the model requests more thinking, the response is long, the prompt contains many words. Model self-confidence can be useful telemetry.
It should not be the only trigger.
External failure is a better trigger than internal uncertainty Imagine two tasks.
Task A The model says:
I am only 60% confident.
But the deterministic test suite passes every required acceptance test.
There may be no reason to escalate.
Task B The model says:
I am highly confident this is correct.
But the integration test fails.
The runtime should escalate.
Therefore:
model confidence < external verification evidence whenever external evidence exists.
A simple escalation contract Represent escalation decisions explicitly.
from dataclasses import dataclass, field from enum import Enum class Escalation(str, Enum): NONE = "none" MORE_EVIDENCE = "more_evidence" MORE_SAMPLES = "more_samples" STRONGER_MODEL = "stronger_model" SPECIALIST = "specialist" SEARCH = "search" ADVERSARIAL_REVIEW = "adversarial_review" HUMAN = "human" @dataclass class RuntimeSignals: verified: bool = False verification_unknown: bool = False failed_checks: int = 0 disagreement: float = 0.0 progress_delta: float = 0.0 route_confidence: float = 1.0 repeated_failures: int = 0 risk: float = 0.0 @dataclass class EscalationDecision: action: Escalation reason: str estimated_cost: float = 0.0 evidence: list[str] = field(default_factory=list) The runtime should be able to answer:
Why did this task receive additional compute?
If the answer is only “the model wanted to think more,” the policy is not very inspectable.
Deterministic escalation rules are a strong baseline Do not jump immediately to an LLM deciding which LLM to call next.
A simple controller may outperform a learned router.
def choose_escalation(s: RuntimeSignals) -> EscalationDecision: if s.verified: return EscalationDecision( action=Escalation.NONE, reason="required verification passed", ) if s.failed_checks > 0: return EscalationDecision( action=Escalation.MORE_EVIDENCE, reason="external verification failed", evidence=[f"failed_checks={s.failed_checks}"], ) if s.disagreement > 0.35: return EscalationDecision( action=Escalation.MORE_EVIDENCE, reason="candidate disagreement is high", evidence=[f"disagreement={s.disagreement:.2f}"], ) if s.route_confidence < 0.6: return EscalationDecision( action=Escalation.SPECIALIST, reason="expert routing confidence is low", ) if s.repeated_failures >= 2: return EscalationDecision( action=Escalation.STRONGER_MODEL, reason="cheap path failed repeatedly", ) if s.verification_unknown and s.risk > 0.7: return EscalationDecision( action=Escalation.HUMAN, reason="high-risk task lacks sufficient verification evidence", ) return EscalationDecision( action=Escalation.NONE, reason="no escalation trigger fired", ) This controller is simple.
That is a feature.
You now have a baseline against which a learned escalation policy must prove itself.
Escalation should be specific A common mistake is:
failure ↓ use a bigger model But not every failure is a model-capacity failure.
Different failures imply different escalations.
Failure signal Better escalation Missing factual evidence retrieval / tool call Candidate disagreement more evidence or diverse samples Wrong specialist reroute expert Shallow reasoning failure deeper reasoning Premature branch pruning wider search / MCTS Repeated tool failure recovery strategy, not more prose Critic finds defect targeted revision Verification unavailable gather evidence / human review Local model cannot complete task frontier-model escalation This is why adaptive compute is not just model routing.
It is mechanism routing.
Local model → frontier model cascades One of the most useful real-world patterns is:
local model ↓ external verification ├── PASS → stop └── FAIL / UNKNOWN ↓ frontier model ↓ verify again This gives local models the easy workload while reserving expensive models for cases where they can add value.
A more advanced cascade can be:
deterministic tool ↓ local model ↓ local specialist ↓ frontier generalist ↓ frontier specialist / search But each level should have a measurable reason to exist.
Adaptive reasoning depth Reasoning itself can be budgeted dynamically.
Instead of:
always generate 20 reasoning steps use:
reason ↓ check whether uncertainty decreased ↓ continue only if another step could change the decision One simple metric is decision change.
Ask:
Did the additional reasoning produce a new hypothesis, new evidence request, new constraint, or different action?
If not, continued reasoning may just be token expansion.
Track:
reasoning steps useful state changes new evidence requests verified-success delta A long reasoning trace is not evidence that the compute was useful.
Adaptive sampling Self-consistency does not require a fixed sample count.
Start with a small number:
N = 2 or 3 Measure agreement.
If the answers converge and external evidence supports the result, stop.
If they disagree materially, sample more or gather stronger evidence.
def next_sample_count(current_n: int, agreement: float) -> int: if agreement >= 0.9: return current_n if agreement >= 0.7: return min(current_n + 2, 7) return min(current_n * 2, 10) Again, the important signal is not disagreement alone.
It is whether additional samples improve the final verified decision.
Adaptive search width Tree search can also change width dynamically.
Use a narrow frontier when one branch is clearly superior under strong evidence.
Use a wider frontier when scores are close or uncertain.
clear evidence ↓ beam width 1–2 uncertain evidence ↓ beam width 4–8 This is preferable to fixing a large beam width for every task.
You can define:
def beam_width(score_margin: float, evaluator_variance: float) -> int: if score_margin > 0.4 and evaluator_variance < 0.1: return 2 if score_margin > 0.2: return 4 return 8 The exact numbers are application-specific.
The principle is general:
Spend search compute where the frontier is ambiguous.
Adaptive MCTS budgets MCTS is especially expensive if treated as a default.
A better strategy is:
beam search first ↓ measure shallow-score reliability ↓ if promising branches have unstable / delayed value ↓ allocate MCTS budget MCTS should be an escalation triggered by evidence that greedy or beam-like pruning is unreliable.
Possible triggers:
low rank stability across evaluator samples, high pruning regret in similar historical tasks, delayed objective signals, long-horizon action consequences, weak correlation between shallow score and final verified result. This is much stronger than:
Complex task → run MCTS.
Verification-driven escalation The cleanest adaptive architecture is often organized around the verifier.
attempt ↓ verify ├── PASS → return ├── FAIL → diagnose → targeted escalation └── UNKNOWN → gather more evidence / escalate The verifier becomes a control signal.
That means verification is no longer merely the final gate.
It also helps decide how much more work is warranted.
Failure type should control the next action Suppose a coding agent fails.
The naive policy is:
retry the model The adaptive policy asks what failed.
Syntax failure Use parser/compiler evidence.
Test failure Inspect the failing test and affected code path.
Missing repository context Retrieve relevant symbols/files.
Wrong architecture assumption Escalate to broader repository reasoning or specialist review.
Repeated patch failure Try alternative strategies or search.
High-risk migration Require stronger verification or human review.
The recovery mechanism should match the failure.
Avoid escalation loops Adaptive agents introduce a new failure mode:
fail ↓ escalate ↓ fail ↓ escalate ↓ fail ↓ escalate forever Every adaptive system therefore needs hard budgets.
Track at least:
max escalation levels max model calls max tool calls max wall time max tokens max monetary cost max search nodes And return an explicit termination reason.
class Termination(str, Enum): VERIFIED_SUCCESS = "verified_success" VERIFIED_FAILURE = "verified_failure" BUDGET_EXHAUSTED = "budget_exhausted" NO_PROGRESS = "no_progress" HUMAN_REQUIRED = "human_required" EVIDENCE_UNAVAILABLE = "evidence_unavailable" A budget exhaustion is not a success.
It is a named outcome.
Budget as first-class state Keep compute budget in runtime state.
@dataclass class ComputeBudget: model_calls_left: int tool_calls_left: int token_budget: int cost_budget: float search_nodes_left: int Every escalation consumes budget.
The controller can then ask:
Is the expected value of another escalation worth the remaining budget?
This is much more useful than an unbounded while not done: loop.
Expected value of escalation A production runtime can estimate whether escalation is worthwhile.
Suppose historical telemetry says:
local model verified success: 82% frontier escalation success: +9 percentage points average extra cost: $0.11 average extra latency: 4.5 s For low-value tasks, escalation may not be worth it.
For a deployment-changing action, it may be trivial compared with the cost of failure.
Conceptually:
expected value = probability escalation changes outcome × value of correct outcome − escalation cost − latency penalty − operational risk You do not need perfect monetary estimates.
Even rough task classes are better than pretending every request has identical value.
Risk-aware compute allocation Risk should influence escalation.
Compare:
"rename this local variable" with:
"apply this production database migration" Even if model confidence is identical, the second task deserves stronger verification.
A useful policy separates:
difficulty risk uncertainty They are not the same thing.
A simple task can be high risk.
A difficult task can be low risk.
Compute policies should consider both.
Confidence is useful only if calibrated If an escalation controller uses confidence, measure it.
Suppose the router reports:
confidence = 0.9 Does that correspond to roughly 90% verified success?
If not, the number is decorative.
Track calibration curves:
predicted confidence bucket vs actual verified success Useful metrics include:
Brier score expected calibration error false-confidence rate false-uncertainty rate But even a calibrated model confidence should remain secondary to direct environment evidence when available.
Adaptive routing between experts The previous post on agent-level Mixture of Experts introduced expert routing.
Adaptive agents add another dimension:
route ↓ execute ↓ observe outcome ↓ reroute if evidence says current expert is insufficient For example:
code formatter ↓ FAIL static analyzer ↓ evidence insufficient local coding model ↓ tests fail frontier coding specialist The route changes because the state changed.
This is more powerful than classifying the task once at the beginning.
Learning from historical success Adaptive policies can start deterministic.
Later, historical data may improve them.
Record:
task features initial route verification result escalation sequence models used experts used search budget cost latency final verified result Then ask:
For tasks like this, which escalation usually changes failure into verified success? That can eventually support learned routing.
But this introduces an important distinction:
Memory records what happened. Learning changes future policy because of what happened.
The next posts in this series will go deeper into that boundary.
A complete adaptive-agent skeleton Here is a compact provider-agnostic runtime.
from dataclasses import dataclass, field from enum import Enum from typing import Any, Callable class Verdict(str, Enum): PASS = "pass" FAIL = "fail" UNKNOWN = "unknown" class ComputeLevel(str, Enum): CHEAP = "cheap" DEEP_REASONING = "deep_reasoning" MULTI_SAMPLE = "multi_sample" SPECIALIST = "specialist" SEARCH = "search" FRONTIER = "frontier" @dataclass class Verification: verdict: Verdict evidence: list[str] = field(default_factory=list) failed_checks: list[str] = field(default_factory=list) @dataclass class Attempt: level: ComputeLevel output: Any verification: Verification cost: float = 0.0 latency_ms: int = 0 @dataclass class Budget: max_attempts: int = 5 max_cost: float = 1.00 class AdaptiveAgent: def __init__( self, policies: dict[ComputeLevel, Callable[[dict], Any]], verifier: Callable[[dict, Any], Verification], ): self.policies = policies self.verifier = verifier def next_level(self, attempts: list[Attempt]) -> ComputeLevel | None: if not attempts: return ComputeLevel.CHEAP last = attempts[-1] if last.verification.verdict == Verdict.PASS: return None if last.level == ComputeLevel.CHEAP: if last.verification.verdict == Verdict.UNKNOWN: return ComputeLevel.MULTI_SAMPLE return ComputeLevel.DEEP_REASONING if last.level == ComputeLevel.DEEP_REASONING: return ComputeLevel.SPECIALIST if last.level == ComputeLevel.MULTI_SAMPLE: return ComputeLevel.SPECIALIST if last.level == ComputeLevel.SPECIALIST: return ComputeLevel.SEARCH if last.level == ComputeLevel.SEARCH: return ComputeLevel.FRONTIER return None def run(self, task: dict, budget: Budget) -> tuple[Any, list[Attempt]]: attempts: list[Attempt] = [] total_cost = 0.0 while len(attempts) < budget.max_attempts: level = self.next_level(attempts) if level is None: break output = self.policies[level](task) verification = self.verifier(task, output) attempt = Attempt( level=level, output=output, verification=verification, ) attempts.append(attempt) total_cost += attempt.cost if verification.verdict == Verdict.PASS: return output, attempts if total_cost >= budget.max_cost: break best = attempts[-1].output if attempts else None return best, attempts This skeleton is intentionally simple.
A production system would make next_level() depend on richer evidence.
But the architecture is visible:
attempt ↓ verify ↓ choose next compute policy ↓ repeat within budget Do not treat the escalation ladder as mandatory order The sample implementation above uses a simple ordered ladder for clarity.
Real systems should not always do:
cheap → reasoning → specialist → search → frontier Sometimes the correct response to failure is immediately:
run a deterministic tool or:
retrieve missing evidence or:
ask for human authorization The runtime should select the smallest mechanism that addresses the diagnosed failure.
Coding-agent application A coding agent can use this cascade:
repository state ↓ deterministic inspection ↓ local coding model ↓ tests / lint / type check ├── PASS → stop └── FAIL ↓ retrieve failing symbols + history ↓ local specialist / deeper reasoning ↓ verify ↓ frontier model only if needed ↓ search / multi-agent review for hard cases Useful escalation signals:
failing tests, repeated patches to same lines, no reduction in failing test count, conflicting static-analysis results, broad architectural changes, high-risk migration paths. Do not use vector-memory uncertainty when git diff, the compiler or test runner can tell you the exact state.
Research-agent application A research agent can begin cheaply:
query ↓ retrieve primary sources ↓ extract claims ↓ claim/source verification Escalate when:
sources disagree, key claims lack primary evidence, retrieved sources are stale, multiple interpretations remain plausible, the claim is high consequence. Possible escalation:
broader retrieval → specialist source search → independent synthesis samples → adversarial claim review More prose is not necessarily more research.
More evidence coverage is.
Customer-support application Support tasks vary dramatically.
Cheap path:
identify customer ↓ retrieve ticket/order state ↓ deterministic policy lookup ↓ local response generation Escalate when:
policy conflict exists, refund authority is required, customer state is inconsistent, issue repeats after remediation, legal/safety constraints are implicated. The adaptive agent may route to:
billing specialist technical specialist fraud system human supervisor The escalation should follow the failure or risk signal.
Data and analytics application For a data agent:
schema inspection ↓ SQL generation ↓ query validation ↓ run against safe environment ↓ result checks Escalate when:
schema ambiguity exists, query plan is unexpectedly expensive, result invariants fail, data quality checks disagree, destructive transformations are requested. Possible advanced mechanisms:
specialist SQL expert query-plan analyzer alternative query search frontier model for semantic ambiguity human approval for destructive writes Browser-agent application A browser agent should use direct page state as its primary signal.
observe DOM ↓ choose action ↓ execute ↓ verify page-state transition Escalate if:
element is missing, expected URL/state did not change, modal/permission state is unexpected, repeated clicks do not progress, irreversible submission is imminent. Do not escalate merely because the page HTML is large.
Escalate because the current policy cannot reliably determine the next safe action.
DevOps and incident-response application Adaptive compute is especially useful in incident systems.
Start with deterministic telemetry:
alerts metrics logs health checks recent deploys Then use models to synthesize hypotheses.
Escalate when:
hypotheses remain tied, the blast radius is increasing, rollback safety is uncertain, evidence contradicts the current diagnosis, the next action is high risk. Possible escalation:
more diagnostics specialist service model Tree of Thoughts adversarial incident review human incident commander Again:
Evidence determines compute allocation.
Application matrix System Cheap path Escalation signal Advanced path Coding local model + tests tests fail / no progress specialist, frontier, search Research primary retrieval source conflict / missing evidence broader retrieval, debate Support state + policy lookup authority/policy conflict specialist or human Data schema + deterministic validation invariant/query-plan failure SQL specialist, search Browser DOM + action verification repeated no-progress stronger planner/specialist DevOps telemetry + runbook uncertain/high-risk diagnosis search, specialist, human The adaptive mechanism is the same.
The trigger is domain-specific.
Metrics for adaptive agents You cannot optimize adaptive compute if you only record final success.
Track:
verified success rate first-pass success rate escalation rate escalation success delta unnecessary escalation rate missed escalation rate average compute level model calls per task tool calls per task latency per task cost per task cost per verified success budget-exhaustion rate human-escalation rate Two particularly useful metrics are:
Escalation precision Of tasks that were escalated, how many genuinely benefited?
beneficial escalations ---------------------- all escalations Escalation recall Of tasks where a more expensive mechanism would have fixed the failure, how many did the controller escalate?
These reveal different problems.
Low precision means you are wasting compute.
Low recall means you are leaving capability unused.
Oracle escalation Offline, run the expensive path even when production would not.
Then ask:
Did the expensive mechanism produce a verified improvement? This creates an oracle escalation label.
Now you can measure:
controller decision vs whether escalation would actually have helped That is a much stronger basis for training or tuning an adaptive router than subjective labels such as “hard task.”
Escalation regret Define escalation regret as cases where:
cheap path failed expensive path would have succeeded controller did not escalate Also measure the opposite:
cheap path already succeeded controller escalated anyway Call that wasted escalation.
Together they characterize the controller.
Failure attribution When an adaptive system fails, ask where.
Capability failure No available policy could solve the task.
Trigger failure A useful escalation existed, but the runtime did not detect the need.
Routing failure Escalation occurred, but to the wrong mechanism.
Budget failure The correct mechanism existed but the budget stopped too early.
Verification failure The runtime had the correct answer but failed to recognize it—or accepted a wrong one.
Control-loop failure The system repeatedly escalated without gaining evidence or progress.
This decomposition is essential.
Otherwise every failure gets blamed on “the model.”
A controlled experiment Do not deploy adaptive compute because it sounds efficient.
Benchmark it.
Compare:
A. cheap model only B. frontier model only C. fixed advanced stack D. deterministic local → frontier cascade E. verification-driven adaptive controller F. learned controller, if justified Measure:
verified success median latency p95 latency average cost cost per verified success escalation rate wasted escalation escalation regret budget exhaustion The adaptive system should earn its complexity.
A useful target curve You want something like:
most tasks ↓ cheap path succeeds some tasks ↓ one targeted escalation few tasks ↓ expensive search / frontier / multi-agent very few tasks ↓ human intervention If every task reaches the expensive tier, the adaptive controller is not doing much.
If no task reaches it, you may be leaving capability on the table.
The deeper principle: intelligence is partly compute allocation The advanced techniques in this series can now be viewed together.
Chain of Thought → allocate compute across intermediate reasoning Self-Consistency → allocate compute across independent samples Tree of Thoughts → allocate compute across partial branches MCTS → adaptively reallocate compute using accumulated branch value Mixture of Experts → allocate work across specialized capabilities Planner–Executor–Critic → allocate responsibility across roles Adversarial Review → allocate review compute toward unresolved claims Adaptive Agents → decide when any of those mechanisms should run That is the unifying architecture.
Advanced agents are increasingly systems for deciding:
where should the next unit of computation go? But adaptive does not mean self-modifying An adaptive runtime changes what it does within or between tasks based on current evidence.
That is different from changing its underlying policy permanently.
adaptive execution ≠ learning from previous runs A runtime that chooses a frontier model after a local failure is adaptive.
A runtime that notices over thousands of tasks that local model X performs poorly on migration work and permanently updates its routing policy is learning.
That distinction matters.
Because once the system begins changing future policy from historical outcomes, new questions appear:
Which outcomes are trustworthy training signals? How do you avoid learning from verifier bugs? How do you prevent bad episodes from poisoning policy? How do you evaluate policy changes before deployment? How do you roll back a learned routing change? That is where the series goes next.
Final rule A sophisticated agent does not run every sophisticated mechanism on every task.
It begins with the cheapest plausible path.
It watches the environment.
It measures failure, uncertainty, risk and progress.
Then it allocates additional compute only where that compute has a reason to change the outcome.
The rule is simple:
Start cheap. Escalate on evidence. Verify every escalation. Stop when the evidence is sufficient.
That is adaptive agency without uncontrolled complexity.
Next Advanced Agents From First Principles 09: Can Your Agent Actually Learn From Previous Runs? Turn Verified Trajectories Into Better Future Policies Without Poisoning the System.