# Advanced Agents From First Principles 21: What Happens When Too Many Agents Compete for the Same Resources? Add Admission Control, Quotas and Backpressure
# Book: Advanced Agents From First Principles
# Chapter URL: https://programmer.ie/books/advanced-agents-from-first-principles/21-chapter/
# Prose fingerprint (sha256, normalized): 9e3fb158933a7407f48c5312482395881e48ced00e44009bea8d0b4fee17cea7
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

What Happens When Too Many Agents Compete for the Same Resources? A single agent can look healthy in isolation.
It gets a request.
It calls a model.
It launches a few search branches.
It opens a browser.
It runs tests.
It asks a verifier to check the result.
Everything works.
Then production traffic arrives.
Ten agents start at once.
Then fifty.
Then five hundred.
Now every agent still has a perfectly reasonable local plan.
Collectively, the platform can become unusable.
The same GPU queue is saturated.
The browser pool is exhausted.
A shared database connection pool reaches its limit.
An external API starts returning 429s.
Long-running research jobs occupy every worker.
High-priority incident-response agents sit behind low-priority batch work.
Speculative search consumes the model capacity that verification needed.
Retries amplify the overload.
Queues grow faster than they drain.
Latency explodes.
And eventually the system reaches a state where every individual component is “trying harder” while the platform as a whole is getting worse.
This is the next layer of agent engineering.
The central rule is:
A production agent platform must schedule demand globally, not merely optimize each run locally.
The previous posts gave us increasingly sophisticated per-run control:
agent run ↓ route ↓ search ↓ observe uncertainty ↓ spend compute dynamically ↓ verify Then we distributed the work:
run ↓ coordinator ↓ workers ↓ leases / fencing / idempotency But none of that answers this question:
Which runs should be allowed to consume shared resources right now?
That is a platform scheduling problem.
Local Optimization Can Destroy Global Performance Suppose each agent has a dynamic budget scheduler.
Each scheduler sees uncertainty and decides that another model call would probably help.
Individually, that can be correct.
Across 1,000 concurrent runs, it can be disastrous.
run A -> another model call run B -> another model call run C -> another model call ... run N -> another model call The local policy knows about its own budget.
It may not know that the shared inference cluster is already at 98% capacity.
This gives us two distinct control layers:
per-run control plane decides how a run spends its allowance platform control plane decides how much allowance the run receives now Do not merge them conceptually.
A run may want more compute.
The platform may correctly refuse.
The Resource Model Is Multi-Dimensional An agent platform rarely has one bottleneck.
It has several.
For example:
resource envelope ├── frontier model requests ├── local model GPU slots ├── tokens per minute ├── browser sessions ├── sandbox workers ├── database connections ├── repository worktrees ├── CPU execution slots ├── memory ├── outbound HTTP concurrency ├── external API rate limits ├── verification capacity └── money A run might be cheap in one dimension and expensive in another.
A research agent may consume little CPU but enormous model and retrieval capacity.
A coding agent may consume moderate model capacity but many sandbox/test workers.
A browser agent may be bottlenecked almost entirely by browser sessions and third-party rate limits.
A data agent may be constrained by database queries and memory rather than inference.
So the scheduler must reason about a resource vector, not one generic concurrency number.
A compact representation might look like this:
from dataclasses import dataclass @dataclass(frozen=True) class ResourceRequest: model_calls: int = 0 gpu_slots: int = 0 browser_slots: int = 0 sandbox_slots: int = 0 db_connections: int = 0 outbound_requests: int = 0 verifier_slots: int = 0 expected_cost_usd: float = 0.0 The exact fields will differ by system.
The architectural point is the same:
Resource demand should be explicit enough that the platform can reason about contention before work starts.
Admission Control Comes Before Queueing A common mistake is to accept every task and put it into a queue.
That sounds safe because a queue appears to provide buffering.
But an unbounded queue is often just delayed failure.
If requests arrive faster than the system can complete them, then:
arrival rate > service rate means:
queue length -> infinity in the idealized case.
In production it usually means:
latency becomes unacceptable, jobs become stale before they start, memory or storage fills, deadlines are missed, retries create more demand, operators lose visibility into what is actually runnable. Admission control asks a more basic question:
Should this work enter the active system at all right now?
Possible outcomes are not merely accept or reject.
They may include:
ADMIT_NOW QUEUE DEFER DOWNGRADE SHED REJECT For example:
an interactive coding request may be admitted immediately; a nightly repository scan may be queued; a speculative benchmark may be deferred; a research task may be downgraded to a cheaper model; low-value background work may be shed entirely during an incident. That is not failure.
That is load management.
A Simple Admission Controller Start simple.
Do not begin with an LLM deciding whether the platform has capacity.
Ordinary software is usually enough.
from dataclasses import dataclass from enum import Enum class AdmissionDecision(str, Enum): ADMIT = "admit" QUEUE = "queue" DEFER = "defer" REJECT = "reject" @dataclass(frozen=True) class Capacity: model_slots_free: int browser_slots_free: int verifier_slots_free: int @dataclass(frozen=True) class RunRequest: priority: int needs_model: bool needs_browser: bool needs_verifier: bool deadline_ms: int | None = None def admit(req: RunRequest, capacity: Capacity) -> AdmissionDecision: if req.needs_verifier and capacity.verifier_slots_free <= 0: return AdmissionDecision.QUEUE if req.needs_browser and capacity.browser_slots_free <= 0: return AdmissionDecision.QUEUE if req.needs_model and capacity.model_slots_free <= 0: if req.priority >= 100: return AdmissionDecision.QUEUE return AdmissionDecision.DEFER return AdmissionDecision.ADMIT This is deliberately boring.
That is good.
You can test it.
You can inspect it.
You can reason about overload behavior.
You can evolve it later when evidence demonstrates that the simple policy is insufficient.
Quotas Prevent Noisy Neighbors Suppose one tenant launches 1,000 deep-research jobs.
Without quotas, that tenant can consume all available inference capacity.
Every other user suffers.
This is the classic noisy-neighbor problem.
Agent systems make it worse because one user request can fan out into many internal operations.
For example:
1 user task ↓ 4 candidate generators ↓ 8 search branches ↓ 3 specialist agents per branch ↓ 2 verifiers The external request count may be one.
The internal demand may be dozens or hundreds of resource-consuming operations.
Quota accounting must therefore happen at the correct layer.
Useful quota dimensions include:
per tenant per user per project per repository per workload class per model tier per external integration And possible quota units include:
concurrent runs model calls per minute tokens per hour browser-minutes sandbox CPU-seconds verification calls daily spend Quota Is Not the Same as Rate Limiting These ideas overlap but solve different problems.
A rate limit usually constrains velocity:
requests / second A quota may constrain accumulated or concurrent consumption:
100 concurrent tasks $50/day 1M tokens/hour You often need both.
For example:
rate limit: max 20 model calls / second quota: max 100,000 model calls / day The first protects immediate capacity.
The second protects longer-term allocation and cost.
Fairness Is a Scheduling Policy, Not a Feeling If multiple groups compete for scarce capacity, what does fair mean?
First-come-first-served is one answer.
It is not always a good one.
Suppose the queue contains:
A: 3-hour background repository analysis B: 2-second interactive code completion C: 10-minute research task D: 5-second incident diagnostic Pure FIFO can make D wait behind A.
That may be operationally unacceptable.
So we need an explicit fairness model.
Common approaches include:
FIFO within a class, strict priority classes, weighted fair queuing, deficit round robin, tenant shares, deadline-aware scheduling, shortest-remaining-processing-time approximations, combinations of these. There is no universal best scheduler.
The important rule is:
Make the trade-off explicit and measurable.
Weighted Fairness Imagine three workload classes:
interactive: weight 5 production automation: weight 3 background analysis: weight 1 Under contention, the scheduler can distribute service roughly according to those weights.
This does not mean interactive work gets unlimited resources.
It means it receives a larger share while other classes still make progress.
A simplified allocation could look like:
weights = { "interactive": 5, "automation": 3, "background": 1, } def shares(total_slots: int) -> dict[str, int]: total_weight = sum(weights.values()) result = {} for name, weight in weights.items(): result[name] = max(1, total_slots * weight // total_weight) return result Real schedulers need to deal with unused shares, bursts, minimum guarantees and heterogeneous resource types.
But the core idea is useful:
Fairness should survive load, not disappear exactly when capacity becomes scarce.
Priority Without Starvation Strict priority queues are tempting.
P0 before P1 P1 before P2 P2 before P3 But a constant stream of P0 work can starve P2 forever.
That can be disastrous for maintenance, indexing, memory compaction, benchmark evaluation and other background functions.
Aging is one solution.
The longer a job waits, the more effective priority it gains.
For example:
effective_priority = base_priority + waiting_time_bonus This provides urgency without permanent starvation.
Another approach is reserved capacity.
For example:
80% dynamic shared pool 10% background maintenance reserve 10% verification reserve Reserved capacity can look inefficient during light load.
Under stress it may be what keeps the platform correct.
Verification Needs Platform-Level Protection We have repeatedly protected verification inside a single agent run.
The same principle applies globally.
Suppose every agent reserves local verification budget.
That still does not help if the global verifier pool has been consumed by speculative work from other runs.
So the platform may need a protected verifier lane.
shared capacity ├── generation/search pool ├── tool-execution pool └── protected verification pool Why?
Because the worst overload failure is not merely slow responses.
It is a system that can still generate actions but no longer has enough capacity to establish whether those actions were correct.
That creates false completion pressure.
The platform rule should be:
Speculative work may wait. Required verification must retain a path to completion.
This does not imply infinite verification capacity.
It means verification competes under a different policy than optional exploration.
Backpressure Means Saying “Not Yet” Consider a pipeline:
request ↓ planner ↓ model inference ↓ browser/tool execution ↓ verification ↓ commit Suppose verification can process 100 tasks per minute.
Upstream generation produces 500 tasks per minute.
If generation continues unrestricted, the verification queue grows endlessly.
Backpressure propagates the bottleneck upstream.
verifier saturated ↑ reduce tool execution ↑ reduce branch expansion ↑ reduce admission This is essential.
Without backpressure, every subsystem optimizes its own throughput and pushes the problem downstream.
With backpressure, the platform recognizes that downstream capacity limits upstream usefulness.
Backpressure Is Different From Failure A saturated downstream resource may be healthy.
It is simply busy.
The correct response may be:
WAIT rather than:
RETRY IMMEDIATELY Immediate retries during overload make the problem worse.
This is retry amplification again, now at platform scale.
A backpressure-aware client may receive something like:
@dataclass(frozen=True) class CapacitySignal: available: bool retry_after_ms: int | None queue_depth: int saturation: float The calling layer can then reduce or postpone work instead of blindly retrying.
Queue Depth Is Not Enough A queue of 1,000 tiny jobs may be healthy.
A queue of 20 jobs that each require 30 minutes of GPU time may be disastrous.
So queue depth alone is weak.
Better signals include:
estimated queued work oldest job age p50 wait time p95 wait time service rate arrival rate resource utilization deadline miss probability per-class backlog One useful abstraction is work seconds.
Instead of:
queue length = 50 estimate:
queued GPU work = 3,200 GPU-seconds That gives the scheduler a more realistic view of backlog.
Of course estimates will be imperfect.
That is still often better than pretending every job is equal.
Workload Classes Matter Not all agent work has the same operational semantics.
A useful platform may classify work into categories such as:
interactive production automation incident response verification background indexing offline benchmark speculative research maintenance Each class can have different:
admission thresholds, queue limits, resource shares, deadlines, retry behavior, model tiers, preemption rules, verification requirements. For example:
interactive coding low latency target moderate compute ceiling immediate admission preference background benchmark high latency tolerance large total compute allowance defer under contention incident response high priority protected diagnostic capacity aggressive deadline verification protected reserve cannot be silently skipped This is much better than treating all jobs as generic “agent requests”.
Should Jobs Be Preempted? Sometimes a high-priority task arrives while all workers are busy.
One option is preemption.
pause / cancel low-priority work ↓ free capacity ↓ run urgent task This can be useful.
It can also be dangerous.
Preemption is easiest when work is:
checkpointable, idempotent, side-effect free, resumable, cheap to restart. It is much harder when a worker is halfway through an irreversible external operation.
So a job should expose preemption safety explicitly.
@dataclass(frozen=True) class JobMetadata: workload_class: str priority: int checkpointable: bool side_effecting: bool safe_to_preempt: bool The scheduler should not infer this from a model’s confidence.
It should be part of the runtime contract.
Reservation Versus Opportunistic Capacity Some work needs predictable capacity.
Other work can consume leftovers.
A useful architecture separates:
reserved capacity guaranteed for critical classes shared capacity allocated fairly opportunistic capacity consumed only when idle For example:
GPU cluster ├── 20% verification / critical reserve ├── 60% shared production pool └── 20% opportunistic benchmark/search pool If the shared production pool is quiet, the opportunistic pool may borrow capacity.
When production demand rises, that borrowed capacity is reclaimed.
This lets expensive experiments use idle hardware without compromising production guarantees.
Bursting Hard quotas can waste capacity.
Suppose tenant A has a 20% share but no current work.
Tenant B has a burst of requests.
It can be reasonable to let B temporarily exceed its nominal share.
This is bursting.
But bursting should be reclaimable.
Otherwise temporary borrowing becomes permanent domination.
A common policy is:
guaranteed share + burst credits or token-bucket style allocation.
For example:
class TokenBucket: def __init__(self, capacity: float, refill_per_second: float): self.capacity = capacity self.tokens = capacity self.refill_per_second = refill_per_second def allow(self, cost: float) -> bool: if self.tokens < cost: return False self.tokens -= cost return True A real implementation also needs time-based refill and concurrency safety.
The point is conceptual:
Allow bursts without surrendering long-term fairness.
Deadline-Aware Scheduling Some tasks have explicit deadlines.
For example:
respond to a user within 10 seconds, diagnose an incident before a failover window closes, verify a deployment before the rollout continues, finish a nightly job before business hours. A scheduler can use deadlines as part of priority.
But be careful.
If a task cannot possibly finish before its deadline, aggressively prioritizing it may just waste scarce resources.
A better scheduler can estimate feasibility.
estimated remaining work vs remaining deadline If success is no longer plausible, possible actions include:
downgrade scope return partial result switch to cheaper/faster policy cancel mark deadline miss That is more useful than consuming the entire cluster on an impossible deadline.
Cost-Aware Scheduling Capacity is not only physical.
External model APIs may scale almost infinitely from your perspective but have financial cost.
So the platform may have a global spend envelope.
hourly spend ceiling daily spend ceiling per-team budget per-project budget This can interact with priority.
For example:
high-value production task allowed expensive frontier model low-priority background task local model only during budget pressure The scheduler may therefore choose not only when work executes, but which resource tier it receives.
request ↓ platform scheduler ├── local model ├── small hosted model ├── frontier model └── defer This is related to routing, but the decision is driven by shared platform economics rather than merely task capability.
Rate Limits Are External Capacity Constraints Your platform may be healthy while a dependency is saturated.
Examples:
GitHub API rate limit, search API quota, browser target throttling, database QPS limit, model-provider RPM/TPM limits. Treat these as first-class resources.
Do not hide them inside tool errors.
capacity state ├── internal GPU available: yes ├── browser slots available: yes ├── GitHub API budget: low └── verifier capacity: healthy Then scheduling can adapt before failures occur.
For example:
GitHub budget low ↓ prefer cached repository state ↓ coalesce duplicate reads ↓ defer background scans ↓ reserve remaining calls for interactive/verification work This is much better than letting every worker independently discover the rate limit by receiving errors.
Coalescing Duplicate Work At platform scale, many agents may request the same information at the same time.
For example:
20 agents -> fetch same repository tree 30 agents -> retrieve same documentation page 15 agents -> query same model metadata Instead of executing all calls independently, the platform can coalesce them.
same cache key / operation identity ↓ one in-flight request ↓ multiple subscribers This is sometimes called single-flight behavior.
It reduces:
external API pressure, cost, latency variance, duplicate work. But only when the operation is safe to share.
Do not coalesce requests that depend on tenant-private state or mutable transactional context unless the cache key captures that scope correctly.
Head-of-Line Blocking A shared queue can create another problem.
Suppose the first queued job needs:
2 GPU slots + 4 browser slots but only one browser slot is free.
Behind it are ten jobs that only need a model slot and could run immediately.
If the scheduler blindly preserves queue order, everything waits.
That is head-of-line blocking.
A resource-aware scheduler can scan for runnable jobs while preserving fairness constraints.
queue ├── A needs unavailable browser capacity ├── B runnable now ├── C runnable now └── D runnable now Run B/C/D if policy allows, while ensuring A does not starve forever.
Again, scheduling is a multi-resource problem.
The Scheduler Needs Backpressure From Real Utilization Agent schedulers should not rely only on static configuration.
They need runtime signals.
Useful signals include:
GPU utilization GPU memory pressure model queue depth model p95 latency browser pool occupancy sandbox startup latency database pool saturation external 429 rate verification backlog worker lease churn retry rate queue age But avoid reacting too aggressively to noisy measurements.
If the scheduler changes policy every few seconds, you can create oscillation.
high load -> throttle load falls -> release throttle load rises -> throttle ... Hysteresis helps.
For example:
enter overload mode at 90% leave overload mode below 70% This prevents constant flipping around one threshold.
Overload Modes A production agent platform benefits from explicit overload states.
For example:
NORMAL CONSTRAINED SEVERE EMERGENCY Each state can activate deterministic policy changes.
NORMAL full routing options background work allowed CONSTRAINED reduce speculative width defer new background work SEVERE cheap models first suspend benchmarks preserve interactive + verification EMERGENCY critical workloads only strict admission no optional search This is much easier to reason about than hundreds of independent thresholds changing behavior invisibly.
And again:
Known operational safety policy should remain deterministic unless strong evidence justifies something more complex.
Backpressure Should Reach the Agent Policy The global scheduler should not merely queue the run while the run’s internal policy behaves as if capacity were infinite.
A capacity signal can be exposed to the per-run scheduler.
For example:
@dataclass(frozen=True) class PlatformPressure: model_pressure: float browser_pressure: float verifier_pressure: float external_api_pressure: float overload_mode: str Then the run can adapt within allowed boundaries.
Example:
model pressure high ↓ reduce Best-of-N from 8 to 3 ↓ use deterministic diagnostics first ↓ escalate only when evidence supports it But there is an important boundary.
The local agent should not be allowed to override global quotas simply because it believes its task is important.
run requests resources platform grants or denies Authority remains with the platform scheduler.
Fairness Metrics If you claim your scheduler is fair, measure it.
Useful metrics include:
per-class throughput per-tenant throughput per-class wait time p95 queue age starvation incidents quota rejection rate share utilization borrowed capacity preemption rate deadline miss rate You can also track whether actual resource shares match configured weights over a suitable window.
For example:
configured share: 30% actual 1-hour share: 58% That may be fine during bursting.
It may indicate broken fairness if sustained during contention.
Backpressure Metrics Measure whether overload is propagating correctly.
For example:
queue growth rate admission rejection/defer rate retry-after compliance upstream fan-out reduction external 429 rate verification backlog resource saturation duration A useful question is:
When one downstream resource saturates, does upstream demand fall?
If not, your backpressure mechanism is probably decorative.
Scheduling Failure Taxonomy Trajectory observability should include platform-level failure labels.
For example:
ADMISSION_OVERLOAD QUOTA_EXCEEDED STARVATION PRIORITY_INVERSION HEAD_OF_LINE_BLOCKING VERIFICATION_STARVATION RATE_LIMIT_EXHAUSTION RETRY_AMPLIFICATION RESOURCE_FRAGMENTATION PREEMPTION_FAILURE BACKPRESSURE_FAILURE SCHEDULER_OSCILLATION DEADLINE_MISS BUDGET_EXHAUSTION This makes incidents diagnosable.
Instead of:
agent timed out we might get:
FAIL └── DEADLINE_MISS ├── queue_wait_ms = 14200 ├── model_wait_ms = 3100 ├── verifier_wait_ms = 8200 └── root cause = verification pool saturation That is a platform problem, not a prompt problem.
Distributed Scheduling Needs Idempotency Too Step 20 introduced leases and fencing for distributed task execution.
The platform scheduler itself must respect those semantics.
A queued run might be:
admitted, assigned, worker lost, requeued, assigned again. That must not create duplicate side effects.
So scheduling integrates with:
operation identity attempt identity lease epoch checkpoint identity fencing token The scheduler decides where work runs.
The ownership layer decides whether that worker still owns the attempt.
The commit boundary decides whether the attempt may mutate shared state.
Do not collapse these responsibilities into one giant coordinator object.
Platform Scheduler Events Extend the event taxonomy.
Useful events include:
RUN_SUBMITTED RUN_ADMITTED RUN_QUEUED RUN_DEFERRED RUN_SHED RUN_STARTED RUN_PREEMPTED RUN_RESUMED QUOTA_CHECKED RESOURCE_RESERVED RESOURCE_RELEASED BACKPRESSURE_APPLIED OVERLOAD_MODE_CHANGED DEADLINE_AT_RISK RATE_LIMIT_PRESSURE Each event should carry enough context for later analysis.
For example:
{ "event": "RUN_QUEUED", "run_id": "r-1024", "workload_class": "interactive", "priority": 80, "reason": "model_pool_saturated", "queue_depth": 34, "estimated_wait_ms": 1800, "scheduler_version": "sched-v7" } Do not log hidden model reasoning.
Log operational state and explicit scheduler decisions.
Scheduling Policies Should Be Versioned The scheduler is part of the control plane.
Treat it like code.
For example:
scheduler_version: sched-v7 classes: incident: weight: 10 max_queue: 50 reserved_model_slots: 4 interactive: weight: 5 max_queue: 1000 automation: weight: 3 max_queue: 5000 background: weight: 1 max_queue: 10000 defer_in_overload: true verification: reserved_fraction: 0.15 overload: constrained_at: 0.80 severe_at: 0.90 emergency_at: 0.97 Then policy changes are:
diffable, reviewable, replayable, testable, reversible. That is much safer than embedding scheduling behavior across dozens of agents.
Test the Scheduler With Synthetic Load Do not wait for a real overload incident.
Create load tests.
Examples:
Test 1: Noisy neighbor tenant A submits 10,000 background jobs tenant B submits 10 interactive jobs Expected:
tenant B retains service, tenant A is quota-limited or queued, verification remains available. Test 2: Verification saturation verifier latency increases 10x Expected:
upstream speculative fan-out reduces, verification reserve remains protected, admission becomes stricter, false completion does not increase. Test 3: External API rate limit GitHub allowance drops to 5% remaining Expected:
background repository scans defer, duplicate reads coalesce, interactive/verification work retains access. Test 4: Priority flood continuous high-priority traffic Expected:
lower classes slow down, but aging/reservations prevent permanent starvation where required. Test 5: Retry storm model provider returns 503 for 30 seconds Expected:
retries back off, admission reduces, queues remain bounded, service recovers without synchronized retry spikes. Compare Scheduling Policies Fairly Like every advanced mechanism in this series, the scheduler must earn its complexity.
Compare policies under the same workload traces.
For example:
A: FIFO B: strict priority C: weighted fairness D: weighted fairness + admission control E: D + dynamic backpressure Measure:
verified task success p50 latency p95 latency p99 latency cost throughput starvation queue age deadline misses external 429s verification backlog Do not declare the sophisticated scheduler better merely because it has more mechanisms.
It must improve the outcomes that matter.
Coding-Agent Example Imagine a shared coding platform serving:
interactive bug fixes PR reviews repository indexing nightly refactoring experiments CI repair agents The expensive resources are:
frontier-model calls, repository worktrees, sandbox test workers, GitHub API calls. A reasonable policy may be:
interactive fixes high weight short queue target CI repair high priority during active failures PR review medium weight repository indexing background pause under GitHub pressure nightly experiments opportunistic only If GitHub API capacity falls, the system can:
stop speculative repository scans, reuse cached tree state where valid, reserve remaining calls for active fixes and verification, defer background indexing. The model prompt has nothing to do with this.
This is runtime engineering.
Research-Agent Example A research platform may have:
interactive research questions scheduled reports large literature reviews benchmark crawls source verification The scheduler may protect:
primary-source retrieval, citation verification, user-facing interactive work. Broad speculative retrieval can be reduced under pressure.
For example:
normal mode: query 12 retrieval sources constrained mode: query top 5 severe mode: query 3 authoritative sources first Again, complexity degrades gracefully instead of collapsing abruptly.
Browser-Agent Example Browser pools are often scarce and stateful.
Suppose you have 100 browser sessions.
A few aggressive agents can consume them all through speculative exploration.
So the scheduler may define:
interactive browser tasks: 40 reserved production automation: 40 shared research/discovery: 20 opportunistic If interactive demand spikes, opportunistic exploration is cancelled or deferred.
Side-effecting browser flows may receive stronger isolation and lower concurrency than read-only discovery.
Data-Agent Example Data agents may compete for:
warehouse query slots, database connections, memory, CPU workers. One complex query can consume far more capacity than many small ones.
So queue depth is especially misleading.
Use estimated query cost when possible.
small metadata query cost weight 1 medium aggregate cost weight 10 full historical recomputation cost weight 500 Then weighted admission can protect transactional or interactive workloads from expensive background analysis.
DevOps-Agent Example During an incident, dozens of diagnostic agents may wake up at once.
That is exactly when infrastructure may already be degraded.
Bad policy:
incident detected ↓ launch every diagnostic everywhere Better policy:
incident detected ↓ enter incident workload class ↓ reserve diagnostic capacity ↓ run cheapest/highest-EVI checks first ↓ fan out only if uncertainty remains ↓ protect verifier + authoritative-state reads This connects directly back to the earlier posts on uncertainty and Expected Value of Information.
Global scheduling should not erase those mechanisms.
It should constrain them according to shared capacity.
Global Scheduling + Dynamic Per-Run Budgets Now we can combine the layers.
platform scheduler ↓ grants resource envelope run scheduler ↓ allocates envelope by expected value uncertainty model ↓ identifies what blocks progress EVI policy ↓ selects useful observation speculative executor ↓ runs bounded parallel work distributed coordinator ↓ owns attempts safely verifier ↓ proves result This is a much more realistic architecture than one giant autonomous agent.
Each layer solves a different problem.
And importantly:
Every layer can be simplified or removed if evidence shows it does not earn its cost.
A Compact Platform Scheduler Interface A useful interface might look like this:
from dataclasses import dataclass from enum import Enum class ScheduleAction(str, Enum): START = "start" QUEUE = "queue" DEFER = "defer" DOWNGRADE = "downgrade" SHED = "shed" @dataclass(frozen=True) class PlatformRequest: run_id: str tenant_id: str workload_class: str priority: int deadline_ms: int | None resources: ResourceRequest @dataclass(frozen=True) class ScheduleDecision: action: ScheduleAction reason: str scheduler_version: str granted_resources: ResourceRequest | None = None retry_after_ms: int | None = None Notice what is missing.
No hidden chain-of-thought.
No free-form “agent intuition”.
The scheduler makes an operational decision based on explicit state.
Do You Actually Need a Sophisticated Scheduler? Maybe not.
If you have:
low traffic, one tenant, one model, no meaningful external rate limits, no latency classes, plenty of spare capacity, then a bounded FIFO queue may be completely adequate.
Start there.
Add quotas when noisy neighbors appear.
Add priorities when workload classes matter.
Add reservations when critical work is starved.
Add weighted fairness when multiple groups need guaranteed progress.
Add dynamic backpressure when bottlenecks move around.
The evidence-first progression is:
simple queue ↓ measure failure ↓ add smallest scheduling mechanism ↓ benchmark ↓ keep only if it earns its cost Do not build Kubernetes for three cron jobs.
Final Architecture At this point the advanced agent platform has several nested control loops.
incoming work ↓ PLATFORM SCHEDULER admission / quota / fairness / pressure ↓ granted envelope ↓ RUN CONTROL routing / budget / stopping policy ↓ UNCERTAINTY what blocks progress? ↓ VALUE OF INFORMATION what should we observe next? ↓ SPECULATIVE / PARALLEL WORK ↓ DISTRIBUTED WORKER COORDINATION leases / fencing / retries ↓ SIDE EFFECT GATE ↓ VERIFICATION ↓ PASS / FAIL / UNKNOWN ↓ trajectory + evidence The system is more complex than a single model call.
But the complexity now has explicit responsibilities.
That matters.
The platform scheduler does not reason about code correctness.
The verifier does not decide tenant fairness.
The worker lease does not decide whether MCTS deserves another node.
The model does not decide whether it still owns a distributed mutation.
Those boundaries are what make the architecture understandable.
The Core Rules If you remember only a few things from this post, remember these:
Local agent optimization is not global platform optimization. Admission control is safer than accepting infinite backlog. Resource demand is multi-dimensional. Quotas prevent one workload from consuming the whole platform. Fairness must be explicit and measurable. Priority needs starvation protection. Verification deserves protected platform capacity. Backpressure must propagate upstream. Retries during overload can amplify failure. External rate limits are real capacity constraints. Schedulers should degrade optional complexity before correctness mechanisms. Scheduling policy should be versioned, replayable and reversible. Simple FIFO is often the right starting point. A more sophisticated scheduler must beat the simpler baseline under realistic load. And the most important rule is still the same rule that has guided the entire series:
Add a mechanism because you can identify the failure it is supposed to fix and measure whether it fixes it.
At platform scale, that rule becomes even more important.
Because once hundreds or thousands of agents compete for shared resources, complexity can fail globally even when every individual agent looks locally reasonable.
The next step is to deal with another consequence of operating at that scale:
What happens when dependencies fail partially?
A model provider slows down.
A browser service starts timing out.
A retrieval API becomes flaky.
A verifier cluster is degraded.
A database remains reachable but returns stale replicas.
The next post will look at failure containment, circuit breakers, bulkheads and graceful degradation—how an advanced agent platform keeps one broken dependency from cascading through every run.