# Advanced Agents From First Principles 31: How Can an Agent Learn New Capabilities Without Expanding Its Own Authority? Use Sandboxed Capability Acquisition
# Book: Advanced Agents From First Principles
# Chapter URL: https://programmer.ie/books/advanced-agents-from-first-principles/31-chapter/
# Prose fingerprint (sha256, normalized): 4eaaa553f50057d257830daa3bb9933a342d4ed3a6ad14c2151793b7e34a781a
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

The safest answer to an out-of-distribution task is not always:
refuse forever Sometimes the right answer is:
learn safely first That creates a difficult systems problem.
Step 30 introduced competence envelopes.
An agent may be allowed to observe a task, propose a solution, or experiment in isolation while still being prohibited from applying the result to production.
That is useful.
But if the competence boundary can never move, the system cannot accumulate new validated capability.
If the boundary moves too easily, however, the entire safety architecture collapses.
The agent encounters a novel task.
It succeeds once.
It declares itself competent.
It receives more authority.
That is not learning.
That is self-certification.
The core rule for this post is:
Learn outside the authority boundary before expanding the authority boundary.
A production agent should be able to discover that it may have a new capability.
It should not be able to grant itself the authority that depends on that claim.
That separation gives us a new architecture:
production authority boundary | | task outside competence envelope v isolated capability sandbox | | experiment / search / practice v external verification | | repeated evidence v candidate competence claim | | independent promotion gate v updated competence envelope | v separately updated authority policy The important word is candidate.
A sandbox can produce evidence.
It cannot promote itself.
The Search Problem: “Can an AI Agent Learn New Skills Safely?” A lot of discussion about self-improving agents jumps immediately to dramatic ideas:
recursive self-improvement, autonomous skill creation, agents rewriting their own prompts, agents generating new tools, agents modifying their own policies, agents training themselves from experience. Those ideas bundle several very different operations together.
A safer decomposition is:
explore ↓ produce candidate behavior ↓ measure it ↓ collect evidence ↓ propose a competence change ↓ independent promotion That looks less magical.
Good.
It also makes the process testable.
The question is not:
Can the agent improve itself?
The more useful question is:
Can the platform generate and validate candidate improvements without allowing the candidate system to change its own authority?
That is a much better engineering problem.
Capability, Competence and Authority Are Different Things We now have three separate concepts.
Capability Can the system sometimes perform the task?
Competence Do we have sufficient evidence that the system performs this class of task reliably under defined conditions?
Authority Is the system permitted to perform the task at a particular level of consequence?
They are not interchangeable.
capability does it appear able to do this? competence have we demonstrated reliable performance? authority may it do this here? An agent can possess capability without validated competence.
It can possess validated competence without production authority.
And it can possess production authority only within the policy boundary granted by the platform.
This separation is the foundation of safe capability acquisition.
One Success Is Not a Competence Claim Suppose a coding agent has never modified Rust before.
It receives an unfamiliar Rust task.
The competence system classifies the task as:
UNVALIDATED The platform lets the agent attempt the task inside an isolated repository worktree.
The code compiles.
The tests pass.
The patch is correct.
What did we learn?
We learned:
this agent succeeded on this task We did not learn:
this agent is now reliable on Rust repositories Those are radically different claims.
A competence claim requires evidence across a task distribution.
That usually means some combination of:
repeated trials, varied tasks, adversarial cases, held-out cases, relevant environment variation, strong external verification, false-success measurement, failure-mode coverage, sufficient sample size. The agent can generate evidence.
The evidence system decides what that evidence supports.
Build a Capability Sandbox The sandbox is not merely a filesystem directory.
It is an authority-restricted execution environment.
A useful sandbox should control at least:
filesystem writes network access credentials external APIs production databases message sending payments cloud mutations browser submissions secret access process execution resource budgets The exact controls depend on the workload.
But the architectural principle is consistent:
Experimental capability acquisition must not inherit production authority by accident.
For a coding agent, that might mean:
real repository snapshot ↓ isolated worktree/container ↓ no production credentials ↓ restricted network ↓ compile/test/static analysis ↓ patch artifact For a browser agent:
recorded or synthetic site ↓ test account ↓ no real payment method ↓ no external message delivery ↓ observable interaction trace For a DevOps agent:
production-like topology ↓ local simulator / staging cluster ↓ synthetic failure injection ↓ no production mutation credentials The sandbox should be realistic enough to produce useful evidence while remaining incapable of creating the production consequence being studied.
Sandboxing Is an Authority Property This distinction matters.
A Docker container is not automatically a safe sandbox.
A browser profile is not automatically a safe sandbox.
A Git worktree is not automatically a safe sandbox.
A Kubernetes namespace is not automatically a safe sandbox.
The important question is:
what authority can escape the environment? If the sandbox contains:
production credentials, unrestricted outbound network access, writable production databases, deployment tokens, real payment APIs, messaging credentials, then it may be isolated computationally while remaining dangerous operationally.
A capability sandbox therefore needs an explicit authority manifest.
For example:
from dataclasses import dataclass from typing import Literal Authority = Literal[ "read_fixture", "write_workspace", "execute_local", "network_read_allowlist", "external_mutation", ] @dataclass(frozen=True) class SandboxPolicy: sandbox_id: str allowed_authorities: frozenset[Authority] network_allowlist: tuple[str, ...] max_runtime_seconds: int max_model_calls: int max_tool_calls: int max_cost_usd: float environment_hash: str For capability acquisition, external_mutation should normally be absent.
The Acquisition Loop A minimal capability-acquisition loop looks like this:
unvalidated task family ↓ select sandbox experiment ↓ execute candidate approach ↓ external verification ↓ record result + provenance ↓ update evidence set ↓ enough evidence? no ───────────────┐ ↓ | design next test | └────────────┘ ↓ yes candidate competence claim ↓ independent promotion review Notice what is missing.
There is no step called:
agent updates its own authority That operation belongs outside the acquisition loop.
Treat Capability Learning as Experimental Design Once capability acquisition is isolated from authority, the problem starts looking much more like experimental science.
That is useful.
You have:
a hypothesis, an intervention, observations, external measurements, repeat trials, controls, evidence strength, promotion criteria. Suppose the hypothesis is:
H1: This behavioral release can reliably repair small Rust ownership errors in repositories with deterministic compile/test verification. That hypothesis is much better than:
The agent learned Rust. The first claim has boundaries.
The second is almost meaningless.
Define the Candidate Competence Claim Before Testing Avoid moving the goalposts after seeing results.
Define the proposed expansion first.
For example:
from dataclasses import dataclass @dataclass(frozen=True) class CandidateCompetenceClaim: claim_id: str release_id: str task_family: str environment_class: str tool_requirements: tuple[str, ...] verifier_requirements: tuple[str, ...] maximum_authority: str maximum_risk_class: str minimum_verified_success: float maximum_false_success: float maximum_unknown_rate: float minimum_trials: int This prevents a common failure mode:
run experiments ↓ observe mixed results ↓ redefine competence to match the wins The intended claim should exist before the benchmark evidence is interpreted.
Separate Development Evidence From Promotion Evidence This is crucial.
The same examples used to discover a technique should not be the only examples used to certify it.
A simple split is:
development set discover candidate capability calibration set tune policies / thresholds held-out promotion set evaluate competence claim production shadow/canary validate transfer The exact terminology can vary.
The principle should not.
If the agent repeatedly sees the same tasks while adapting, success on those tasks is evidence of memorization or local adaptation as much as competence.
You need fresh evidence.
The Agent Can Propose Experiments The agent can still be extremely useful inside this architecture.
It can propose:
new approaches, new prompts, new tool sequences, new decompositions, new search strategies, new candidate skills, new verifier combinations, new test cases, new failure hypotheses. For example:
@dataclass(frozen=True) class CapabilityExperimentProposal: proposal_id: str competence_claim_id: str hypothesis: str procedure_ref: str sandbox_policy_id: str verifier_plan: tuple[str, ...] expected_information_gain: str estimated_cost: float That preserves the agent’s ability to explore.
But the experiment runner—not the candidate agent—enforces the environment and authority constraints.
Do Not Let the Agent Choose Only Easy Evidence An adaptive agent will naturally discover regions where it performs well.
That is useful for capability discovery.
It is dangerous for capability certification.
If the same system chooses the tasks on which it will be evaluated, it may gradually construct a flattering evidence set.
You need independent sampling.
A promotion benchmark should include:
randomly sampled eligible tasks, difficult edge cases, known historical failures, adversarial variants, scale variation, environment variation, tool failures, verifier stress cases. The agent may generate additional tests.
It should not control the entire test distribution.
Positive Evidence Is Not Enough Competence acquisition needs negative evidence too.
Store:
successes failures false successes UNKNOWN outcomes timeouts unsafe proposals verifier disagreement resource overruns OOD subclusters The failures define the edge of the envelope.
An agent that succeeds on 95 ordinary cases and fails catastrophically on five high-risk edge cases may not deserve the proposed authority expansion.
This is why the Step 27 reliability vector matters.
Do not reduce acquisition evidence to one success percentage.
Capability Evidence Should Be Typed A useful evidence record might look like:
from dataclasses import dataclass from typing import Literal Outcome = Literal[ "PASS", "FAIL", "UNKNOWN", "FALSE_PASS", "POLICY_VIOLATION", ] @dataclass(frozen=True) class CapabilityTrial: trial_id: str claim_id: str release_id: str task_descriptor_hash: str environment_hash: str sandbox_policy_id: str candidate_hash: str verifier_bundle_id: str outcome: Outcome cost_usd: float latency_ms: int provenance_ref: str The result should be replayable.
The candidate artifact should be immutable.
The verifier bundle should be versioned.
The environment should be identified.
Otherwise the evidence will decay into anecdote.
Verification Must Stay Outside the Candidate A capability-acquisition loop has an obvious reward-hacking risk.
If the candidate system can decide whether its own experiment succeeded, the easiest route to apparent improvement is to weaken the test.
So:
candidate generation ≠ acceptance authority Use external verification whenever possible.
For coding agents:
compiler, tests, type checker, linter, static analysis, security scanners, property tests, integration tests. For research agents:
source existence, source authority, quote/reference matching, date validation, claim-evidence binding, independent source diversity. For browser agents:
simulated postconditions, DOM state, synthetic account state, transaction simulator, replayable browser traces. For DevOps agents:
health checks, invariant checks, recovery objectives, synthetic load, fault-injection outcomes. The stronger the external verifier, the stronger the competence evidence.
Never Let the Learner Rewrite the Verifier Silently Suppose an agent repeatedly fails a test.
It proposes changing the test.
That may be legitimate.
The test may really be wrong.
But the change must become a separate candidate artifact.
candidate behavior change ↓ verifier unchanged or candidate verifier change ↓ independent verifier-release process Do not bundle them casually.
Otherwise:
agent fails benchmark ↓ agent weakens benchmark ↓ agent passes ↓ "capability improved" That is not capability acquisition.
It is measurement corruption.
Step 24’s independent verifier release gate applies here directly.
Use Search to Discover Capability Advanced search techniques are useful inside the sandbox.
You can use:
self-consistency, beam search, tree search, MCTS, evolutionary search, planner/executor/critic loops, mixtures of specialists, prompt variants, tool-policy variants. But remember the series rule:
Complexity is not a power ladder.
If a simple deterministic procedure solves the new task family reliably, prefer it.
Capability acquisition can discover that the right answer is not a smarter agent.
It may discover:
new deterministic parser new compiler check new schema validator new routing rule new static analyzer new retrieval index new test fixture That is still capability growth.
Often it is better capability growth.
Capability Acquisition Is a Search Over Systems The candidate being evaluated does not need to be only a prompt.
It may be a complete behavioral release variant:
model prompt router search policy budget policy critic policy tool set retrieval configuration memory behavior verifier bundle This means the search space can become enormous.
Do not explore everything at once.
Start with the failure evidence.
Ask:
what specific mechanism appears insufficient? Then vary that mechanism while holding the rest stable when feasible.
That preserves attribution.
Use Step 18: Expected Value of Information Capability acquisition can consume enormous compute.
Do not blindly run thousands of experiments.
Ask which experiment is most likely to change the competence decision.
Suppose the current evidence says:
ordinary cases: strong large repositories: weak evidence adversarial inputs: unknown verifier failures: unknown The next useful experiment is probably not another ordinary case.
It is one that reduces uncertainty at the competence boundary.
The acquisition scheduler can reuse Expected Value of Information:
candidate experiment ↓ which competence uncertainty does it reduce? ↓ how likely is it to change promotion decision? ↓ what does it cost? ↓ run highest-value experiment This is active learning at the systems level.
Target the Boundary, Not the Center If you already know the agent performs well on trivial tasks, repeating them adds little evidence.
The most informative experiments often sit near the edge of the claimed competence envelope.
For example:
repository size small ───────── medium ───────── large PASS PASS ? Or:
risk class read-only reversible bounded mutation irreversible PASS PASS ? EXCLUDED Or:
source quality primary strong secondary noisy web adversarial PASS PASS ? FAIL The boundary tells you what to test next.
But Do Not Optimize Only for Boundary Expansion There is a subtle trap.
If the acquisition system is rewarded for expanding the envelope, it may prefer optimistic interpretations of weak evidence.
The objective should not be:
maximize competence area It should be closer to:
maximize useful validated capability subject to reliability and authority constraints Contraction remains valid.
The system may learn that a previously broad competence claim should become narrower.
That is useful learning too.
Competence Can Become More Precise Instead of Larger Suppose you start with:
LIMITED: Python repository maintenance After experiments, you discover:
VALIDATED: - dependency upgrades with lockfile verification - local refactors under 500 LOC - deterministic test-backed bug fixes LIMITED: - cross-service changes UNVALIDATED: - database migrations EXCLUDED: - security-critical authentication changes without human review The envelope did not simply expand.
It became better shaped.
That is often more valuable.
Build Capability Curricula From Evidence A useful acquisition system can create a curriculum.
Not a generic difficulty ladder.
An evidence-driven curriculum.
For example:
known competence ↓ nearest unsupported task cluster ↓ sandbox experiments ↓ verified local competence ↓ next boundary cluster This is different from assuming:
easy → medium → hard Task difficulty is multi-dimensional.
A seemingly simple task may introduce:
new tools, new authority, weak verification, new environment assumptions, unusual side effects. The curriculum should follow evidence gaps, not labels like beginner/intermediate/advanced.
Skills Should Be Versioned Artifacts If the acquisition process discovers a reusable procedure, represent it explicitly.
For example:
@dataclass(frozen=True) class SkillArtifact: skill_id: str version: str task_family: str procedure_hash: str required_tools: tuple[str, ...] required_verifiers: tuple[str, ...] allowed_authority: str evidence_set_id: str release_compatibility: tuple[str, ...] A skill could be:
a deterministic workflow, a prompt template, a routing policy, a tool sequence, a retrieval strategy, a planning template, a verifier composition. The word skill should not mean an opaque blob of learned behavior.
It should mean a versioned artifact with evidence.
Keep Skill Discovery Separate From Skill Installation This is the same authority separation again.
agent discovers candidate skill ↓ artifact stored ↓ benchmark evidence attached ↓ review / promotion gate ↓ skill registry update The agent should not be able to write directly into the production skill registry merely because a candidate worked in the sandbox.
The registry is a production control surface.
Treat it like one.
Promotion Should Be Evidence-Gated A capability promotion can follow the release states from Step 24.
For example:
DISCOVERED ↓ SANDBOX_VALIDATED ↓ HELD_OUT_VALIDATED ↓ SHADOW ↓ LIMITED_AUTHORITY ↓ CANARY ↓ VALIDATED And at any stage:
BLOCKED REJECTED ROLLED_BACK This is intentionally boring.
Boring is good when the alternative is autonomous self-certification.
A Candidate Promotion Record from dataclasses import dataclass @dataclass(frozen=True) class CompetencePromotionProposal: proposal_id: str claim_id: str release_id: str evidence_set_id: str current_state: str proposed_state: str current_max_authority: str proposed_max_authority: str verified_success_rate: float false_success_rate: float unknown_rate: float sample_size: int verifier_bundle_id: str known_failure_signatures: tuple[str, ...] unresolved_gaps: tuple[str, ...] Notice again:
This is a proposal.
The promotion engine applies the policy.
The candidate agent does not.
Authority Expansion Should Lag Competence Evidence Even after competence improves, authority need not expand immediately.
That is useful.
For example:
competence: VALIDATED for task family authority: proposal only The organization may intentionally leave the system there.
Why?
Because technical competence is not the only constraint.
There may be:
legal requirements, organizational policy, customer expectations, financial exposure, security boundaries, audit requirements. Competence answers whether the system is reliable.
Authority answers whether it may act.
Keep them separate even after learning succeeds.
Use Human Review at the Promotion Boundary, Not Every Trial If humans must approve every sandbox experiment, capability acquisition becomes painfully slow.
That usually is not necessary.
The better architecture is:
many low-authority sandbox trials ↓ automated external verification ↓ structured evidence aggregation ↓ one promotion decision Human attention is concentrated at the point where authority or production behavior changes.
This is the same principle from Step 29:
Use human review where authority changes, not as a substitute for machine-verifiable work.
Sandboxed Coding-Agent Example Suppose a coding agent is validated for Python but not Go.
A Go issue arrives.
Production decision:
competence = UNVALIDATED production mutation authority = denied Capability acquisition can still proceed.
1. snapshot repository 2. create isolated worktree 3. remove production credentials 4. allow compiler/test tooling 5. let agent inspect and patch 6. run gofmt 7. run go test 8. run static analysis 9. record patch + trace + evidence 10. destroy workspace Over time, the platform collects a benchmark set:
small bug fixes 18/20 verified API refactors 13/20 verified dependency upgrades 20/20 verified concurrency changes 7/20 verified security changes insufficient evidence A sensible competence update might be:
SUPPORTED: Go dependency upgrades with deterministic verification LIMITED: small Go bug fixes UNVALIDATED: API refactors EXCLUDED FROM AUTONOMOUS AUTHORITY: concurrency-critical and security-critical changes That is useful learning without pretending the agent “learned Go” globally.
Research-Agent Example Suppose a research agent is validated on public primary sources but encounters a new domain: regulatory filings in a jurisdiction it has never processed.
Sandbox acquisition can:
retrieve archived filings, ask the agent to extract claims, compare against known structured records, test citation accuracy, test date interpretation, inject conflicting filings, measure false claims, test missing-data behavior. The system may discover that the agent performs well only when:
filing type is known source is primary OCR quality is high claim requires no jurisdiction-specific legal interpretation That becomes the competence boundary.
Not:
agent can research regulation Browser-Agent Example A browser agent is validated for read-only browsing and simple form filling.
The platform wants to explore travel booking.
Do not give it a real credit card and see what happens.
Build a synthetic or test environment.
Test:
multi-step forms, price changes, session expiry, confirmation pages, duplicate submissions, cancellation flows, unexpected upsells, stale DOM state, ambiguous buttons. Most importantly, test side-effect discipline.
Does the agent distinguish:
prepare booking ≠ submit booking Does it stop at the commit boundary?
Does it preserve the correct artifact for approval?
Does it revalidate price and state before commit?
These are capability properties too.
DevOps-Agent Example A DevOps agent is competent at rolling service restarts.
You want to explore database failover recovery.
Use a staging topology or simulator.
Inject:
node failure, stale replica, network partition, delayed metrics, conflicting health checks, lease expiry, operator interruption. Measure:
recovery correctness, data integrity, unsafe action proposals, time to recover, whether the agent recognizes uncertainty, whether it escalates when invariants become unverifiable. A successful recovery is not enough.
The agent must also demonstrate that it knows when not to execute recovery autonomously.
That is part of competence.
Capability Acquisition Can Improve Escalation Sometimes the learned capability is not:
solve the task autonomously It is:
recognize this task earlier collect the right evidence prepare a better human handoff That can be enormously valuable.
Suppose the agent repeatedly fails to solve a particular database incident safely.
But it learns to gather:
topology state, replication lag, current leader, pending writes, health history, recent deployments, relevant logs. Then the capability improvement may be:
A1 → A1 better Better proposal and diagnosis capability without expanded mutation authority.
That is still a meaningful improvement.
Reward Safe Abstention If capability acquisition rewards only successful task completion, the agent may learn to push ahead when it should stop.
The evaluation should include correct abstention.
For example:
safe task + success → positive unsafe task + escalation → positive unverifiable task + UNKNOWN → positive unsafe task + confident action → severe negative false PASS → severe negative This prevents a dangerous learning objective:
maximize completion rate Completion is not the same thing as competence.
Learn the Boundary as Well as the Procedure A good capability artifact has two parts:
how to perform the task and when this procedure is valid For example:
@dataclass(frozen=True) class SkillApplicability: task_family: str supported_environments: tuple[str, ...] required_tools: tuple[str, ...] required_verifiers: tuple[str, ...] excluded_risk_classes: tuple[str, ...] maximum_scale: str | None known_failure_signatures: tuple[str, ...] Without applicability constraints, skill acquisition tends to become overgeneralization.
The system learns something useful in one regime and applies it everywhere.
That is exactly what competence envelopes are supposed to prevent.
Watch for Capability Leakage A sandbox experiment can leak into production in subtle ways.
For example:
sandbox memory enters production retrieval, experimental prompts become default prompts, candidate skills enter the production registry, benchmark answers enter training context, temporary credentials persist, generated tools become globally available, experimental policy weights are reused by production routing. Treat sandbox output as quarantined until promoted.
sandbox artifacts ↓ quarantine store ↓ verification ↓ promotion gate ↓ production registry No implicit path should exist around that gate.
Keep Experimental Memory Separate This is especially important for adaptive agents.
Suppose the sandbox agent learns:
when you see this repository, use workaround X If that memory is immediately visible to production, you have already modified production behavior.
Even if no code was deployed.
So distinguish:
experimental memory production memory Promotion of memory should be explicit too.
A memory entry can be a behavioral artifact.
Treat it that way.
Dataset Contamination Is Capability Leakage Too If held-out promotion cases are later fed into training or memory, future promotion claims may become invalid.
Record dataset lineage.
For each task, know whether it belongs to:
development calibration held-out promotion shadow production incident regression Do not casually mix them.
This is not bureaucracy.
Without data-role separation, the competence evidence becomes impossible to interpret.
Preserve the Scientific Boundary A particularly important rule:
Promotion evidence must remain genuinely independent of the adaptation process it evaluates.
If the agent can inspect, optimize against, or repeatedly retry the exact promotion set, it is no longer held out.
The platform should enforce this mechanically where possible.
For example:
acquisition worker can access development cases promotion evaluator can access held-out cases candidate agent cannot enumerate promotion corpus This is the same reason test-set access matters in ordinary machine learning.
Agent systems do not get an exemption merely because the adaptation happens through prompts and workflows instead of gradient descent.
Capability Acquisition Needs Budgets Exploration can become an infinite compute sink.
Give it explicit budgets.
@dataclass(frozen=True) class AcquisitionBudget: max_trials: int max_model_calls: int max_tool_calls: int max_cost_usd: float max_wall_clock_seconds: int max_parallelism: int And preserve Step 16’s principle:
max budget ≠ target budget Stop when additional experiments no longer have enough expected decision value.
Stop Rules Matter Capability acquisition should stop when:
the claim is sufficiently supported, the claim is clearly rejected, evidence is no longer informative, the budget is exhausted, a hard safety failure occurs, verifier coverage becomes insufficient, the experimental environment diverges too far from the target environment. Do not continue experimenting merely because the agent can generate another variation.
Negative Stop Conditions Some failures should block promotion immediately.
Examples:
unauthorized mutation attempt sandbox escape attempt credential exfiltration attempt systematic verifier bypass repeated false PASS unbounded retry behavior non-idempotent duplicate side effects in simulation The exact policy depends on risk.
But severe safety failures should not disappear into an average score.
Promote the Smallest Claim Supported by Evidence Suppose experiments support:
reliable CSV schema repair Do not promote:
data engineering competence Suppose the agent reliably repairs:
React component prop-type errors Do not promote:
frontend engineering competence A good competence system prefers narrow, defensible claims.
The envelope can expand again later.
Capability Expansion Should Be Monotonic Only in Evidence, Not Authority There is no reason authority must only increase.
Evidence can increase while authority contracts.
For example, a new benchmark may reveal a failure mode.
The platform learns more.
The correct response may be:
more evidence ↓ narrower competence claim ↓ less authority That is not regression.
It is better calibration.
Version Every Acquisition Decision A competence claim is valid only relative to a behavioral release and its environment assumptions.
Record:
model version, prompt version, tool versions, router policy, search policy, memory schema, verifier bundle, sandbox policy, environment class, benchmark corpus version, promotion policy. If those change materially, the evidence may not transfer automatically.
Step 23’s drift logic applies directly.
Revalidate After Behavioral Releases Suppose you validated a capability under release R17.
Then you change:
the model, the system prompt, the router, the tool schema, the verifier. You now have release R18.
Do not assume:
competence(R17) == competence(R18) Some evidence may transfer.
Some may not.
The competence registry should record compatibility rules and required revalidation.
Acquisition Evidence Can Decay Even without a release change, the environment can change.
For example:
browser sites redesign, APIs change, package ecosystems evolve, infrastructure topologies change, regulatory sources change format, repository conventions shift. Competence evidence therefore has freshness assumptions.
A capability validated two years ago may no longer justify the same authority today.
Competence is not a permanent badge.
Build a Competence Registry A production system can maintain an explicit registry.
@dataclass(frozen=True) class CompetenceRecord: competence_id: str release_id: str task_family: str state: str maximum_authority: str evidence_set_id: str verifier_bundle_id: str benchmark_version: str valid_environment_classes: tuple[str, ...] exclusions: tuple[str, ...] expires_at: str | None The router can consult this before selecting an expert.
The authority gateway can consult it before allowing a mutation.
The acquisition runtime can consult it when deciding what evidence gap to explore next.
Do Not Let Routing Hide Missing Competence A mixture-of-agents system often has a temptation:
choose whichever expert scores highest But if all experts are outside their validated envelopes, the correct route may be:
NO_SUPPORTED_EXPERT Then the platform can choose:
human escalation, sandboxed acquisition, deterministic fallback, refusal, evidence gathering. Do not force a winner from a set of unsupported options.
Acquisition Can Create New Specialists Sometimes repeated sandbox evidence shows that a particular task family deserves a dedicated expert.
For example:
general coding agent ↓ repeated weak performance on SQL migrations ↓ sandbox experiments ↓ dedicated migration workflow performs better ↓ candidate migration specialist That specialist still goes through the same evidence and release process.
The existence of a new agent does not grant it authority.
Acquisition Can Also Eliminate Specialists The reverse can happen.
Suppose experiments show that a specialized critic contributes no measurable improvement over a deterministic check.
The acquisition system can propose:
remove critic Capability acquisition should be allowed to simplify the platform.
Otherwise it becomes a one-way complexity generator.
That would violate the series’ central rule.
Measure Capability Acquisition Itself The acquisition system needs metrics.
Useful ones include:
promotion precision promotion regression rate false competence rate false exclusion rate time to validated competence cost per validated capability experiments per promotion boundary-test yield held-out transfer rate shadow transfer rate canary regression rate authority expansion regret authority contraction latency sandbox policy violation rate verifier bypass rate The most important may be false competence rate.
A false competence claim is dangerous because it converts experimental success into misplaced trust.
Promotion Precision Matters More Than Promotion Volume A system that discovers ten candidate capabilities and safely promotes three may be healthier than one that promotes all ten.
The objective is not:
number of new skills It is:
useful validated capability This distinction becomes critical once capability promotion affects production authority.
Track Predicted Versus Realized Competence After promotion, compare the predicted reliability to production outcomes.
For example:
predicted verified success: 96–98% observed production: 94% predicted false success: <0.5% observed production: 1.4% That evidence should feed back into:
the competence claim, the promotion policy, the benchmark design, future capability-acquisition estimates. The promotion process itself can be calibrated.
Shadow Before Authority A particularly useful stage is production shadow execution.
The candidate system receives real tasks.
It observes the real environment.
It produces candidate actions.
But those actions are not committed.
The production system or human workflow continues normally.
Then compare:
candidate decision vs actual verified outcome Shadow mode gives much stronger transfer evidence than a synthetic benchmark alone.
And because it has no mutation authority, the risk is dramatically lower.
Canary Authority, Not Just Canary Traffic Traditional canaries often mean:
send 1% of traffic to new version For agents, authority can be canaried too.
For example:
100% shadow observation ↓ 20% proposal authority ↓ 5% reversible execution authority ↓ 1% bounded mutation authority The exact percentages are not the point.
The important idea is that traffic exposure and authority exposure are separate dimensions.
That is much more useful for consequential agents.
Promotion Must Respect Error Budgets A candidate capability may pass its local benchmark while the platform’s broader reliability state is poor.
Step 27 gave us error budgets.
Use them.
If the relevant false-success budget is already burning rapidly, do not expand authority merely because the new skill looks promising.
A reasonable policy might be:
HEALTHY → promotion permitted WATCH → shadow / proposal only CONSTRAINED → no authority expansion FREEZE → promotions blocked Capability evidence is necessary.
It is not the only operational condition.
Incident Evidence Can Block Promotion Suppose the candidate competence benchmark looks strong.
But Step 26 incident clustering reveals repeated failures with the same signature in production.
That negative evidence must be included.
A promotion evaluator should ask:
Does this candidate still reproduce known severe incidents? If yes, the competence claim may need to remain limited even if aggregate benchmark performance is high.
Capability Acquisition and Self-Modification Eventually someone will ask:
What if the agent rewrites its own code?
The same architecture still applies.
Self-modification should mean:
agent generates candidate change ↓ isolated build ↓ benchmark ↓ external verification ↓ release artifact ↓ promotion gate It should not mean:
agent edits production runtime ↓ agent restarts itself ↓ hope Self-generated code is just another candidate artifact.
Treat it with the same release discipline as human-generated code.
The Agent Must Not Control the Promotion Policy This boundary should be hard.
The candidate system should not be able to modify:
competence thresholds, false-success ceilings, required verifiers, held-out dataset membership, authority classes, promotion stages, prohibited operations, rollback conditions. Those are governance and reliability controls.
They belong outside the optimized system.
Learned Promotion Policies Need Hard Outer Bounds Could you eventually use a learned model to help prioritize or evaluate promotions?
Possibly.
But even then:
learned promotion advisor ↓ hard deterministic eligibility checks ↓ independent verifier evidence ↓ authority policy The learned system may rank or summarize evidence.
It should not bypass hard constraints.
Separate Evidence Collection From Evidence Interpretation Another useful design split:
trial runner collects artifacts verifier produces outcome evidence evidence aggregator computes metrics promotion evaluator applies policy human reviewer handles exceptional authority decisions This reduces the chance that one component can both create and certify the evidence supporting its own promotion.
Use Provenance Everywhere Every acquisition claim should be traceable.
You should be able to ask:
Why is this capability VALIDATED? and receive something like:
competence claim C-184 ↓ release R-31 ↓ benchmark B-77 ↓ 143 held-out trials ↓ verifier bundle V-12 ↓ 2 false successes ↓ 8 UNKNOWN ↓ shadow cohort S-8 ↓ canary cohort K-3 ↓ promotion policy P-11 ↓ approved state: VALIDATED / A2 That is a defensible competence claim.
Replay Failed Promotions If a capability is promoted and later regresses, Step 25 replay and Step 26 forensics should reconstruct:
what evidence supported promotion, which promotion threshold passed, which benchmark cases existed, which cases were absent, whether verifier behavior changed, whether the workload shifted, whether the authority level was too broad. Then the failed promotion becomes new acquisition evidence.
The system learns not only skills.
It learns where its skill-certification process was weak.
Build Promotion Regression Cases Every false competence incident should produce a durable regression case.
For example:
capability claimed: reliable dependency upgrade incident: lockfile regenerated under incompatible platform new regression case: platform-specific lockfile divergence Future candidate releases must pass that case before reclaiming the same competence claim.
This turns incidents into permanent pressure on the evidence boundary.
Capability Acquisition Is Not Online Learning by Default It is tempting to call all of this online learning.
But many useful implementations require no gradient updates at all.
The system can improve through:
new deterministic workflows, new tool routing, new retrieval indexes, better prompts, better search policies, better verifier composition, new specialists, new competence boundaries, better escalation behavior. That is still learning at the system level.
The architecture does not depend on the model weights changing.
Local Models Make This Especially Interesting If you run local models, sandboxed acquisition can exploit otherwise idle compute.
For example:
production traffic low ↓ acquisition scheduler receives spare GPU budget ↓ run held-out sandbox experiments ↓ cache model responses ↓ verify outcomes ↓ update evidence store But Step 21 still applies.
Acquisition is opportunistic work.
It must yield to production and protected verification capacity.
Do not let self-improvement experiments become the noisy neighbor that damages the service being improved.
Cache Experiments Carefully Repeated capability tests often reuse:
the same repository snapshot, the same prompt, the same model, the same policy, the same tool state. Caching can save huge amounts of compute.
But cache identity must include the behavioral inputs that matter.
At minimum:
model version prompt hash input artifact hash tool schema version policy version environment hash sampling configuration Do not reuse cached results across incompatible behavioral releases and call that fresh evidence.
Synthetic Tasks Have a Role Synthetic tasks can be excellent for:
boundary exploration, failure injection, rare edge cases, systematic parameter sweeps, safety tests. But synthetic success alone should not justify broad production competence.
A useful evidence ordering is often:
synthetic exploration ↓ real held-out historical tasks ↓ shadow production tasks ↓ limited canary authority ↓ production evidence Each stage answers a different question.
Real Historical Failures Are Particularly Valuable Incident cases have one enormous advantage.
We know the old system actually failed there.
So capability acquisition can ask:
Does the candidate system now avoid the original failure? That is stronger evidence than a vague benchmark improvement.
But do not overfit only to incidents.
Keep held-out normal tasks too.
Step 28’s remediation principle applies again.
Never Confuse Simulation Success With Production Equivalence A sandbox necessarily differs from production.
That creates a simulation-to-reality gap.
Record the assumptions.
For example:
sandbox browser uses synthetic checkout production browser uses live payment provider or:
staging DB topology has 3 nodes production has 12 nodes across regions The competence claim should not silently generalize across that gap.
Shadow and canary stages exist for exactly this reason.
Environment Fidelity Is Part of Evidence Strength You can represent evidence strength explicitly.
For example:
E0 synthetic E1 recorded real task E2 live shadow E3 limited canary E4 production verified This is illustrative, not universal.
The point is that one hundred synthetic successes may not be equivalent to ten strongly verified real-world successes.
Evidence quality matters alongside quantity.
Capability Acquisition Can Discover Missing Verifiers Sometimes the result of experimentation is:
we cannot validate this capability safely That is important.
Maybe the agent appears able to perform the task.
But there is no sufficiently strong external verifier.
Then the correct competence decision may remain:
LIMITED or:
UNVALIDATED The next engineering investment should be verifier development, not more agent intelligence.
That follows directly from Step 28’s reliability economics.
Verification Capability Can Be the Bottleneck Imagine an agent that can generate sophisticated database migration plans.
But the platform cannot reliably verify:
data preservation, rollback correctness, lock impact, cross-version compatibility. The agent’s raw capability may exceed the platform’s verification capability.
Production authority should still stop at the verifier boundary.
This gives us an important rule:
Usable competence is bounded by what the platform can verify, not merely by what the model can generate.
Acquisition Can Improve the Verifier First Therefore capability expansion may follow this sequence:
new task family discovered ↓ agent appears capable ↓ verification too weak ↓ build stronger verifier ↓ re-run acquisition trials ↓ collect trustworthy evidence ↓ consider competence promotion This is much safer than promoting capability first and hoping monitoring catches mistakes later.
Distinguish Skill Transfer From Skill Discovery A capability proven under one condition may transfer to another.
But transfer should be tested.
For example:
Python 3.12 ↓ Python 3.13 may require modest revalidation.
Whereas:
Python service ↓ embedded C firmware should not inherit much competence evidence at all.
The transfer policy itself can be explicit and versioned.
Treat Transfer as Another Hypothesis Instead of saying:
skill probably generalizes say:
H-transfer-14: Competence demonstrated on environment E1 transfers to environment E2 without violating SLO X. Then test it.
That keeps generalization evidence-driven.
Cross-Agent Transfer Needs Care Suppose specialist A learns a useful procedure.
Can specialist B use it?
Maybe.
But the evidence belongs initially to:
behavioral system A + skill version S not automatically to:
behavioral system B + skill version S Different models may interpret the same procedure differently.
Different tools may create different failure modes.
Different prompts may change behavior.
Revalidate transferred skills.
Multi-Agent Systems Need Joint Competence Claims A mixture-of-agents system may contain specialists that are individually competent.
The composition can still fail.
For example:
router misroutes specialists disagree aggregator selects wrong answer verifier misses conflict So competence can exist at multiple levels:
specialist competence router competence aggregation competence system competence Capability acquisition should test the composition, not just the parts.
New Skills Can Change Scheduler Economics A candidate capability may use much more compute.
For example:
old workflow: 2 model calls new workflow: 18 model calls + browser + verifier Even if verified success improves, the change may not deserve broad deployment.
Step 12 and Step 28 still apply.
Measure:
verified success, false success, UNKNOWN, latency, cost, verifier load, scheduler pressure. Capability promotion is also an operational release decision.
Compare Against Stronger Single Models Before promoting a complicated learned workflow, test whether a stronger single model solves the same task more cheaply and reliably.
Compare:
candidate acquired skill vs stronger model vs deterministic software vs human escalation The winner should be based on verified outcome and cost—not architectural excitement.
Compare Against Doing Nothing Sometimes the correct answer is still:
keep task out of scope If the workload is rare, high-risk, expensive to verify, and easy to escalate to a human, capability acquisition may have negative return.
That is a legitimate conclusion.
The acquisition runtime should be able to recommend:
DO_NOT_INVEST Capability Acquisition Is Portfolio Management There may be dozens of unsupported task clusters.
Which one should the platform explore next?
Use the same reliability-economics logic from Step 28.
Prioritize by:
workload frequency, expected user value, current failure cost, verifier availability, expected acquisition difficulty, risk class, likely authority level, engineering opportunity cost. The highest-value new capability may not be the most technically impressive one.
A Capability-Acquisition Scheduler At a high level:
class CapabilityAcquisitionScheduler: def choose_next(self, candidates): eligible = [ c for c in candidates if c.sandboxable and c.verifier_available and not c.hard_excluded ] return max( eligible, key=lambda c: c.expected_information_value * c.expected_user_value / max(c.expected_cost, 1e-9), ) Do not mistake the formula for truth.
The useful part is making assumptions explicit.
Real systems should use ranges, uncertainty, and hard constraints rather than fake precision.
Failure Modes Sandboxed capability acquisition introduces its own failure modes.
1. Self-certification agent succeeds agent says it is competent platform expands authority Fix:
Keep promotion external.
2. Benchmark overfitting The agent repeatedly sees the promotion cases.
Fix:
Separate development and held-out evidence mechanically.
3. Sandbox escape Experimental code or tools gain unintended production access.
Fix:
Enforce authority at infrastructure boundaries, not through prompts.
4. Verifier gaming The candidate finds ways to satisfy weak checks without solving the task.
Fix:
Strengthen independent verification and add adversarial cases.
5. Capability leakage Experimental memory, prompts, tools, or policies enter production before promotion.
Fix:
Quarantine experimental artifacts.
6. Overgeneralization A narrow success is promoted into a broad competence claim.
Fix:
Promote the smallest claim supported by evidence.
7. Synthetic overconfidence Simulation results are treated as production evidence.
Fix:
Track environment fidelity and require shadow/canary transfer evidence.
8. Promotion pressure The acquisition system is rewarded for number of skills promoted.
Fix:
Optimize for validated useful capability and penalize false competence.
9. Infinite experimentation The system keeps searching because there is always another variant.
Fix:
Use budgets and information-value stop rules.
10. Authority creep Every competence improvement automatically raises autonomy.
Fix:
Keep competence and authority promotion separate.
Failure Injection Test the acquisition system itself.
Inject situations such as:
sandbox receives a production credential accidentally, candidate attempts disallowed network access, held-out task leaks into development memory, verifier returns an incorrect PASS, candidate modifies verifier configuration, environment hash changes mid-trial, duplicate trial is recorded twice, promotion evidence lacks provenance, release changes after benchmark completion, authority policy attempts to auto-expand, shadow output attempts an external mutation, canary task falls outside proposed cohort, error budget enters FREEZE during promotion. The platform should respond predictably.
Hard Invariants A production capability-acquisition runtime should probably have invariants like:
candidate cannot modify promotion policy candidate cannot modify competence state directly candidate cannot modify authority state directly sandbox cannot hold prohibited production credentials held-out promotion data is inaccessible to adaptation runtime verifier evidence binds to exact candidate artifact missing verification never becomes PASS sandbox outputs are quarantined until promotion promotion requires versioned evidence production authority never exceeds authority policy severe sandbox violations block promotion These should be deterministic checks.
Do not delegate them to an LLM.
A Minimal Architecture You do not need an enormous self-improvement platform on day one.
Start with:
competence registry ↓ unvalidated task queue ↓ isolated sandbox runner ↓ external verifier ↓ evidence store ↓ manual promotion review That already gives you a disciplined learning loop.
Then add only what evidence justifies:
experiment selection, automatic benchmark generation, active boundary testing, shadow execution, canary authority, automated promotion gates, skill registries, learned acquisition policies. Do not begin with the most autonomous version.
A More Complete Architecture At scale:
production task | competence resolver | +------------+------------+ | | in-envelope out-of-envelope | | production runtime acquisition candidate | experiment scheduler | sandbox execution | external verifier | evidence store | candidate competence claim | promotion evaluator | +------------------+------------------+ | | reject shadow/canary | competence update | authority review The important boundaries remain visible.
What This Buys You This architecture gives you a path between two bad extremes.
Extreme one:
agent cannot do unsupported task ↓ permanent refusal Extreme two:
agent encounters unsupported task ↓ tries it live ↓ if it works, assumes competence Sandboxed acquisition gives us:
unsupported ↓ explore safely ↓ measure ↓ repeat ↓ validate ↓ propose promotion ↓ independent authority decision That is a much more credible model of agent learning.
The Deeper Point: Learning Is Not Permission This is the principle worth keeping.
A system can discover a capability without receiving permission to exercise it.
A model can improve without changing the production contract.
An agent can generate a better workflow without installing it.
A specialist can emerge without becoming routable.
A new skill can pass sandbox tests without receiving mutation credentials.
Learning and authority are separate state transitions.
That separation is one of the most important design choices in advanced-agent architecture.
The Complete Loop So Far We now have a much richer runtime:
request ↓ competence check ↓ in-envelope? ────────────── no ──────────────┐ | | yes sandbox acquisition | | route ↓ ↓ external verification plan ↓ ↓ evidence accumulation search ↓ ↓ competence proposal act ↓ ↓ independent promotion verify ↓ ↓ envelope update observe | ↓ | SLOs / drift / incidents <────────────────────┘ And authority remains external throughout.
The Rule If you remember only one thing from this post, remember this:
Let the agent discover capability in a sandbox. Let evidence establish competence. Let an independent policy grant authority. Never collapse those three steps into one.
That is how an advanced agent can become more capable without becoming self-authorizing.
And it gives us the next problem.
Once a system can safely accumulate candidate capabilities, we need to decide what it should learn next.
Not every unsupported task deserves investment.
Not every new skill has equal value.
Not every capability should be optimized merely because it is technically possible.
The next stage is therefore capability portfolio planning:
How should an agent platform choose which missing capabilities to acquire, which to leave to humans or deterministic software, and which to reject permanently?