# Advanced Agents From First Principles 30: Is This Task Outside Your Agent’s Competence? Build Competence Envelopes and OOD Detection
# Book: Advanced Agents From First Principles
# Chapter URL: https://programmer.ie/books/advanced-agents-from-first-principles/30-chapter/
# Prose fingerprint (sha256, normalized): f234c62bc0d7a8082624585db8762692789ca8f793d8f30073692b97c851b0f4
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

Your agent can be confident and still have no business doing the task.
That sounds obvious.
It is not how many agent systems are built.
A task arrives.
The model understands the words.
The router finds a plausible expert.
The tools are available.
The agent produces a plan.
Nothing crashes.
So the system proceeds as though the task belongs to the class of problems the agent can reliably solve.
That assumption is dangerous.
A coding agent that performs well on ordinary Python repository maintenance may be asked to modify a safety-critical embedded controller.
A research agent trained and benchmarked on public web sources may be asked to reason from a sparse, adversarial set of internal documents.
A browser agent that reliably fills ordinary forms may encounter a financial transfer workflow with irreversible consequences.
A DevOps agent that handles routine deployments may be asked to recover a degraded distributed database during a regional outage.
In every case, the agent may be able to produce fluent output.
That is not the same thing as demonstrated competence.
Step 29 gave us authority boundaries and explicit human escalation.
The next question is harder:
How do we know whether the current task is inside the region where this agent has evidence-backed competence at all?
That is the problem of the competence envelope.
The core rule for this post is:
Autonomy should depend on demonstrated competence for the current task regime, not on the agent’s ability to produce an answer.
The Search Problem: “How Do I Know When an AI Agent Is Out of Distribution?” Most production agent systems have a confidence problem.
They ask the model how confident it is.
Or they look at token probabilities.
Or they ask a critic whether the answer looks good.
Or they count how many candidate agents agree.
Those signals may sometimes be useful.
But none of them answers the question we actually care about:
Have we demonstrated that this system is reliable on tasks sufficiently similar to this one, under sufficiently similar operating conditions, with sufficiently strong verification? That is a different question.
A model can be highly confident on an unfamiliar task.
A critic can approve a failure mode it shares with the generator.
Five models can agree because they were trained on similar data.
A nearest-neighbor embedding can say a task is semantically similar while the operational risk is completely different.
The competence envelope therefore cannot be a single confidence score.
It has to be a structured evidence boundary.
Uncertainty Is Not the Same as Competence Step 17 decomposed uncertainty into categories such as:
interpretation uncertainty evidence uncertainty route uncertainty state uncertainty tool uncertainty candidate uncertainty verification uncertainty Those categories describe uncertainty inside a run.
Competence asks a different question:
Should this run be trusted to operate autonomously in this task regime in the first place? You can have:
low uncertainty + low competence evidence For example:
Task: “Rotate the production signing key across all regions.” Agent state: interpretation uncertainty: low evidence uncertainty: low tool uncertainty: low verification uncertainty: moderate Historical evidence: zero production key-rotation cases zero cross-region rollback cases zero benchmark coverage for cryptographic key management The agent may understand exactly what the request means.
It may even know technically correct procedures.
But the system has not demonstrated operational competence for that workload.
That should matter more than confidence.
Define a Competence Envelope A competence envelope is the region of task and operating conditions for which you have enough evidence to grant a particular level of authority.
A useful envelope might include dimensions such as:
CompetenceEnvelope ├── task classes ├── domain classes ├── risk classes ├── environment classes ├── tool families ├── required verifier strength ├── repository / data scale ranges ├── dependency-health assumptions ├── latency / cost operating ranges ├── known failure exclusions ├── minimum evidence volume └── allowed authority level For a coding agent:
validated: Python services < 500k LOC unit + integration tests available ordinary CRUD / API changes no production credential mutation no cryptographic protocol changes repository has clean working tree partially validated: mixed Python / TypeScript migration-heavy changes flaky integration tests not validated: kernel code embedded firmware safety-critical control systems cryptographic primitives The envelope is not a claim that the agent cannot solve tasks outside it.
It is a claim about what the system has evidence to trust.
That distinction matters.
Competence Must Be Authority-Specific Do not ask:
Is the agent competent? Ask:
For this task, with this evidence, what authority level has been justified? Remember the authority classes from Step 29:
A0 — observe A1 — propose A2 — reversible execution A3 — bounded external effect A4 — consequential external effect A5 — privileged / high-blast-radius effect A task may be inside the envelope for A1 but outside it for A4.
For example:
Task: propose a database migration Evidence: strong benchmark coverage schema snapshot available migration linting available Authority: A1 — propose The same agent may not be trusted to execute that migration automatically:
Task: execute database migration in production Evidence: no reliable rollback verifier large blast radius historical production coverage sparse Authority: human-before-commit Competence and authority should therefore be coupled but not conflated.
Start With Task Classification Before checking whether a task is inside the competence envelope, the platform needs to classify the task.
Do not let the model produce one vague label such as:
"coding" That is too coarse.
A useful task descriptor might include:
from dataclasses import dataclass from typing import Literal @dataclass(frozen=True) class TaskDescriptor: domain: str task_family: str operation_kind: str environment: str risk_class: Literal["low", "medium", "high", "critical"] reversibility: Literal["easy", "bounded", "difficult", "irreversible"] external_effects: bool privileged_access: bool estimated_scope: int | None required_tools: tuple[str, ...] required_verifiers: tuple[str, ...] For a coding task:
domain: software_engineering task_family: dependency_upgrade operation_kind: repository_change environment: isolated_worktree risk_class: medium reversibility: easy external_effects: false privileged_access: false required_tools: - repository - package_manager required_verifiers: - unit_tests - dependency_audit For a deployment task:
domain: devops task_family: production_deployment operation_kind: production_mutation environment: prod-eu-west risk_class: high reversibility: bounded external_effects: true privileged_access: true required_tools: - deployment_api - observability required_verifiers: - health_checks - rollout_metrics - rollback_validation The envelope evaluator now has structured dimensions to compare.
Build the Envelope From Evidence, Not Intent A dangerous pattern is:
we designed the agent for this therefore it is competent at this Design intent is not evidence.
The competence envelope should be built from observed performance.
Useful evidence includes:
held-out benchmark results production verified outcomes shadow evaluations canary results incident history failure-injection tests counterfactual replay human-review rescue rates verifier coverage false-success measurements Suppose an agent has:
2,000 verified Python bug-fix runs 500 TypeScript bug-fix runs 40 migration runs 3 production rollback cases 0 cryptographic implementation cases That evidence does not support one universal statement:
agent reliability = 94% It supports several local claims with different strengths.
This is why cohorting from Steps 23 and 27 matters.
Competence Is Local Imagine these benchmark results:
Cohort Verified Success False Success Cases Python bug fix 94% 0.5% 2,000 TypeScript bug fix 90% 0.8% 500 DB migration proposal 87% 1.0% 120 DB migration execution 71% 4.2% 40 Kubernetes incident 62% 7.5% 24 Key rotation UNKNOWN UNKNOWN 0 A single global average destroys the useful information.
The correct conclusion may be:
Python bug fix: autonomous reversible execution permitted TypeScript bug fix: autonomous reversible execution permitted with stronger verification DB migration proposal: proposal permitted DB migration execution: human-before-commit Kubernetes incident: diagnostic support only Key rotation: outside competence envelope That is much closer to how professional systems assign authority.
Out-of-Distribution Is Multi-Dimensional OOD detection is often discussed as though there is one training distribution.
Agent systems have many distributions.
A task can be OOD along one dimension while familiar along another.
Consider:
familiar programming language unfamiliar repository scale familiar operation unfamiliar security context familiar tool unfamiliar tool version familiar task family unfamiliar blast radius familiar environment unfamiliar dependency state A practical OOD vector might look like:
@dataclass(frozen=True) class OODSignals: task_family_novelty: float | None domain_novelty: float | None environment_novelty: float | None tool_novelty: float | None scale_novelty: float | None risk_novelty: float | None verifier_gap: bool unsupported_capability: bool historical_coverage: int But be careful.
These numbers should not become fake probabilities.
They are evidence signals.
If you have not calibrated a value as a probability, do not call it one.
Similarity Is Useful but Dangerous Embedding similarity can help detect novelty.
For example:
new task ↓ embedding ↓ nearest historical task clusters ↓ distance / density / coverage But similarity alone is insufficient.
These two tasks may be semantically close:
restart the staging service restart the production payment service Yet the authority requirements are completely different.
Likewise:
update dependency in test project update dependency in medical-device firmware The code change may look nearly identical.
The operational risk is not.
Therefore:
OOD detection must include operational context, not only semantic similarity.
Use Multiple OOD Signals A robust competence evaluator should combine several types of evidence.
1. Categorical coverage Has this exact task family been benchmarked?
known adjacent unknown 2. Historical density How many comparable verified cases exist?
2 cases ≠ 2,000 cases 3. Distance from known cohorts How far is the current task from validated examples?
4. Environment mismatch Is the current execution environment materially different?
5. Tool mismatch Are new tools, tool versions, or permission scopes involved?
6. Risk mismatch Does the task have greater consequence than the historical cohort?
7. Verification mismatch Is the verifier weaker than the verifier used to establish competence?
8. Distribution drift Has the workload changed since the competence evidence was gathered?
9. Incident exclusions Does the task match a known unresolved failure signature?
No one signal needs to be magical.
The value comes from preserving the dimensions.
Verification Strength Is Part of Competence An agent may appear reliable only because its historical workload had strong verifiers.
For example:
coding cohort unit tests integration tests static analysis type checks may support high autonomous authority.
The same model used for:
open-ended architecture recommendation may have no comparably strong external verifier.
You cannot transfer the same reliability claim.
So competence evidence should record verifier conditions:
@dataclass(frozen=True) class CompetenceEvidence: cohort_id: str release_id: str verifier_profile_id: str verified_success_rate: float | None false_success_rate: float | None unknown_rate: float | None sample_size: int observed_period: str The runtime can then ask:
Is the current verifier profile at least as strong as the verifier profile under which competence was demonstrated? If not, authority may need to contract.
False Success Matters More Than Raw Success Consider two cohorts.
Cohort A verified success: 91% false success: 0.2% UNKNOWN: 5% Cohort B verified success: 95% false success: 6% UNKNOWN: 0.5% If the task is consequential, Cohort A may be far more trustworthy.
This is why competence envelopes must inherit the reliability vector from Step 27.
Do not define competence as:
success_rate > threshold Use something closer to:
verified success acceptable AND false-success ceiling satisfied AND verifier coverage sufficient AND side-effect integrity satisfied AND evidence volume sufficient Define Competence Levels A practical runtime can use discrete states.
For example:
VALIDATED SUPPORTED LIMITED UNVALIDATED EXCLUDED Where:
VALIDATED Strong direct evidence for the task/risk/verifier regime.
SUPPORTED Good adjacent evidence, but not enough for maximum authority.
LIMITED Some useful evidence exists, but human approval or reduced authority is required.
UNVALIDATED No meaningful evidence that the runtime is reliable for this task class.
EXCLUDED Known failure mode, prohibited domain, unresolved incident class, or explicit policy boundary.
Notice that:
UNVALIDATED is not the same as:
FAIL It means:
we do not have enough evidence to trust autonomous execution That is a much more honest systems statement.
A Competence Decision Object Make the result explicit.
from dataclasses import dataclass from typing import Literal CompetenceLevel = Literal[ "VALIDATED", "SUPPORTED", "LIMITED", "UNVALIDATED", "EXCLUDED", ] @dataclass(frozen=True) class CompetenceDecision: level: CompetenceLevel allowed_authority: str matched_cohorts: tuple[str, ...] missing_evidence: tuple[str, ...] ood_reasons: tuple[str, ...] required_verifiers: tuple[str, ...] required_escalation: str | None policy_version: str Now the decision can be logged, replayed, audited, and tested.
It is not hidden inside prompt prose.
Do Not Collapse Everything Into a “Competence Score” It is tempting to write:
competence = 0.82 Then:
if competence > 0.7: proceed That is easy.
It is also dangerous.
Suppose:
task familiarity = high historical coverage = high verifier strength = high risk mismatch = critical A weighted average might still be high.
But one hard risk mismatch should potentially block autonomous execution.
Use hard constraints where hard constraints exist.
For example:
if task.risk_class == "critical" and not evidence.critical_risk_validated: return CompetenceDecision( level="UNVALIDATED", allowed_authority="A1", matched_cohorts=(), missing_evidence=("critical-risk validation",), ood_reasons=("risk class exceeds validated envelope",), required_verifiers=("independent verifier",), required_escalation="dual_control", policy_version="competence-v3", ) The simplicity is a feature.
Hard Boundaries Before Learned OOD Models Before building a neural OOD detector, implement obvious deterministic checks.
Examples:
unknown tool family unsupported file type repository exceeds validated scale privileged operation outside validated environments missing mandatory verifier unseen external system critical task with no critical benchmark cohort known unresolved incident signature These are cheap, explainable, and auditable.
Only add learned novelty models where deterministic structure is insufficient.
This follows the rule we have used throughout the series:
Add complexity only when you can identify the failure the simpler mechanism cannot handle.
Learned OOD Detectors Have Their Own Competence Problem Suppose you train a classifier:
task features → in-distribution / out-of-distribution Now you have another model whose failure can change authority.
That model needs evaluation too.
Important metrics include:
OOD recall false OOD rate missed high-risk OOD rate cohort-specific OOD performance calibration drift The most dangerous failure is often:
OOD task ↓ classified as familiar ↓ authority retained Therefore a learned OOD detector should usually operate inside deterministic authority limits, not replace them.
Distinguish Novelty From Difficulty A familiar task can be difficult.
An unfamiliar task can be easy.
These are different axes.
difficulty low high novelty low routine hard-known high new-easy new-hard The runtime response should differ.
Hard but known You may have evidence that search, stronger models, or escalation works well.
Easy but novel The task may require only proposal mode until evidence accumulates.
Hard and novel This is where autonomy should contract aggressively.
This distinction prevents the scheduler from treating every OOD event as simply “spend more compute.”
More compute does not manufacture competence evidence.
Out-of-Distribution Is Not Fixed Forever The envelope should evolve.
Suppose the agent encounters a new task family:
Rust dependency upgrade Initially:
UNVALIDATED The system may permit:
A0 observe A1 propose human review sandbox execution strong verification After enough evidence accumulates:
100 sandbox cases 50 shadow cases 25 canary cases 0 false successes strong deterministic verification The task family might move to:
SUPPORTED Then later:
VALIDATED This is evidence-gated authority expansion.
Not confidence-gated expansion.
Competence Expansion Should Be a Release Do not let the runtime quietly expand its own envelope because it saw a few successful cases.
Treat envelope changes like behavioral releases.
For example:
candidate envelope change ↓ offline evaluation ↓ shadow ↓ limited authority ↓ canary ↓ promotion The envelope itself should be versioned:
competence_policy: competence-v7 release: agent-release-2026-08-09-4 cohort: python-db-migration previous_level: LIMITED proposed_level: SUPPORTED max_authority: A2 required_verifier_profile: db-migration-v3 minimum_cases: 200 false_success_ceiling: 0.005 Step 24’s release engineering applies here directly.
Competence Can Contract Too The envelope is not monotonic.
Suppose a new incident cluster appears:
large monorepos Python + generated protobuf partial integration-test coverage The system may have previously allowed autonomous A2 execution.
After a false-success spike:
VALIDATED → LIMITED The corresponding authority policy might become:
A2 → A1 + human approval This should happen quickly when risk is high.
Remember the asymmetry from Step 29:
Authority should contract faster than it expands.
Connect Competence to Error Budgets Step 27 introduced SLOs and error budgets.
Competence envelopes should consume that information.
Suppose:
python_bugfix cohort: false-success budget healthy browser_checkout cohort: false-success burn 4x The competence evaluator should not continue treating both cohorts equally.
Possible policy:
HEALTHY retain current envelope WATCH stronger verifier required CONSTRAINED reduce maximum authority FREEZE no autonomous external effects This makes reliability state operational.
Connect Competence to Drift Detection Step 23 covered behavioral drift.
Suppose a task remains inside the nominal envelope, but the environment has drifted.
Examples:
new model version new tool API new browser layout new repository framework new package-manager behavior new deployment topology The task may now be effectively OOD.
So competence should be bound to versions and conditions:
competence evidence model v17 router v8 verifier v5 tool schema v12 environment family E3 Changing those assumptions can invalidate the envelope.
Competence is not a timeless property of the task label.
Connect Competence to Retrieval Research agents need a special form of competence checking.
Suppose the historical benchmark used:
well-indexed public sources high source redundancy recent documents clear provenance Now the runtime receives:
three internal PDFs partial scans conflicting dates missing provenance The topic may be familiar.
The evidence environment is not.
That should produce something like:
evidence_environment_novelty = high verification_gap = true competence = LIMITED The correct response may be:
summarize evidence identify conflicts request missing sources avoid strong factual conclusions rather than simply ask a larger model.
Connect Competence to Browser Agents Browser automation illustrates why semantic familiarity is insufficient.
A browser agent may have excellent evidence for:
search navigation read-only extraction form drafting and very little evidence for:
financial checkout account deletion contract acceptance privilege changes All of these use the same browser.
The tool is familiar.
The authority regime is not.
A browser competence envelope should include:
page/action family external effect type reversibility value at risk identity/account context available postcondition verifier Connect Competence to Coding Agents For coding agents, useful competence dimensions include:
language framework repository size build system test coverage change class security sensitivity runtime environment migration involvement external API impact Example:
Task A: fix null check in Python API Task B: change distributed locking semantics Both are “code changes.”
They should not share an authority envelope.
A useful coding policy might say:
routine localized change strong tests reversible workspace validated language/framework → autonomous candidate + verification cross-service concurrency semantics sparse direct benchmark evidence high blast radius → proposal + isolated experiment + human review Connect Competence to DevOps Agents DevOps competence must be state-sensitive.
The same operation may be familiar under normal conditions and OOD during an incident.
For example:
restart service During normal operation:
healthy dependency graph normal traffic single-region scope known rollback During an outage:
multiple dependencies degraded partial telemetry regional failover active queues saturated The command is the same.
The operating regime is completely different.
So the competence envelope needs environment-state assumptions.
Connect Competence to Multi-Agent Systems Mixture-of-agents systems add another problem.
A router may send an unfamiliar task to the “closest” specialist.
That does not mean any expert is competent.
A good router should be able to return:
NO_SUPPORTED_EXPERT instead of forcing a choice.
For example:
route = router.select(task) if route.coverage == "NONE": return escalate( reason="no expert has validated competence for task regime" ) This is better than:
pick whichever expert has the highest score when every score is poor.
Search Does Not Eliminate the Envelope Tree search, MCTS, beam search, evolution, self-consistency, and debate can all improve candidate exploration.
None of them automatically expands competence.
If the task is OOD, searching more branches gives you:
more outputs in an unvalidated regime That may help.
It may also produce a persuasive failure more efficiently.
Therefore:
OOD ≠ allocate more search by default Use Step 18’s value-of-information logic.
Ask:
What information could move this task back inside a supported envelope? Maybe the answer is:
retrieve authoritative schema run sandbox experiment inspect exact state obtain stronger verifier ask domain expert That is better than blind compute escalation.
A Runtime Decision Pipeline A practical control path might look like:
incoming task ↓ classify task + risk + environment ↓ resolve current release / verifier / tool state ↓ match historical competence cohorts ↓ run hard exclusions ↓ compute OOD evidence vector ↓ check SLO / drift / incident state ↓ derive competence level ↓ derive maximum authority ↓ run / downgrade / escalate / defer / prohibit Notice that the LLM is not the final authority.
The policy engine is.
Example Policy A simple rule system may be enough.
def decide_competence(task, evidence, runtime): if task.operation_kind in runtime.prohibited_operations: return "EXCLUDED", "A0" if evidence.sample_size == 0: return "UNVALIDATED", "A1" if evidence.false_success_rate is None: return "LIMITED", "A1" if task.risk_class == "critical" and not evidence.critical_validation: return "LIMITED", "A1" if runtime.verifier_strength < evidence.required_verifier_strength: return "LIMITED", "A1" if runtime.error_budget_state in {"CONSTRAINED", "FREEZE"}: return "LIMITED", "A1" if evidence.direct_match and evidence.sample_size >= 500: return "VALIDATED", "A2" return "SUPPORTED", "A1" This is intentionally boring.
That is good.
You can always add sophistication after you prove a failure in the simple policy.
UNKNOWN Must Remain a Valid Result Sometimes the system cannot tell whether the task is in distribution.
Maybe:
historical metadata is incomplete cohort labels changed verifier evidence is missing current environment cannot be fingerprinted retrieval provenance is unavailable Do not silently classify the task as familiar.
Return:
COMPETENCE_UNKNOWN and contract authority.
The absence of evidence is not evidence of safety.
Competence Evidence Needs Provenance Step 25’s provenance machinery applies here too.
A competence claim should be traceable to:
benchmark cases production cohorts verifier versions release versions incident exclusions measurement window policy version For example:
cohort: python-api-bugfix level: VALIDATED release: r42 verifier_profile: vp9 sample_size: 1832 window: 2026-06-01..2026-08-01 false_success_rate: 0.003 unknown_rate: 0.021 max_authority: A2 excluded_signatures: - distributed-locking-change - auth-token-rotation Now the statement “this agent is validated for Python API bug fixes” has an evidence trail.
Competence Must Be Recomputed After Significant Releases Step 24 made agent behavior a release bundle.
A major change to:
model prompt router search policy verifier retrieval memory runtime can invalidate old competence evidence.
Do not assume competence transfers automatically.
You may allow evidence carry-over when compatibility has been demonstrated.
But that itself should be explicit.
For example:
old release cohort evidence ↓ compatibility replay ↓ paired benchmark comparison ↓ carry-over approved Otherwise the new release starts with reduced authority.
Use Negative Evidence Competence envelopes should not be built only from successes.
Known failures are extremely valuable.
Suppose incident forensics identified this failure signature:
repository > 1M LOC partial generated code multiple package managers stale dependency graph That should become an explicit exclusion or authority contraction rule.
if signature matches: VALIDATED → LIMITED This is much stronger than hoping the model “learned from the incident.”
Measure Envelope Quality You need to evaluate the competence system itself.
Useful metrics include:
Coverage % of incoming tasks inside VALIDATED/SUPPORTED envelope OOD detection recall % of later-confirmed OOD incidents caught before autonomous execution False OOD rate % of safe familiar tasks unnecessarily downgraded False in-distribution rate % of unvalidated tasks incorrectly granted familiar authority This is especially important.
Escalation rescue rate % of competence-driven escalations where human/stronger process prevented a failure Unnecessary escalation rate % of escalations where autonomous execution would have been verified safe Authority-regret rate cases where granted authority later appears too high Envelope expansion quality newly promoted cohort performance vs expected performance Envelope contraction latency time from drift/incident evidence to authority reduction Test With Deliberate OOD Cases Do not wait for production novelty.
Build OOD test suites.
For a coding agent:
new language new build system very large repository missing tests security-sensitive code concurrency-sensitive code corrupted repository state new tool version For browser agents:
unexpected authentication flow financial transaction page new consent workflow captcha / bot defense ambiguous submit button irreversible account operation For research agents:
single-source evidence contradictory sources stale corpus untrusted documents missing provenance novel domain terminology For DevOps agents:
multi-region outage partial telemetry control-plane degradation stale configuration snapshot unexpected dependency graph privileged emergency operation The test is not simply:
can the agent solve it? The more important test may be:
did the runtime recognize that autonomous authority was not justified? Failure Mode: Confidence Masquerading as Competence The agent says:
I am 95% confident. The runtime proceeds.
This is unacceptable unless that confidence is actually calibrated and relevant to the task regime.
A better rule:
self-confidence can influence search but cannot expand hard authority limits Confidence is an internal signal.
Competence is an evidence claim.
Failure Mode: Nearest Neighbor = Supported The task looks similar to previous tasks in embedding space.
The runtime declares it in-distribution.
But the new task has:
higher risk new tools weaker verification larger blast radius Similarity should produce:
candidate supporting evidence not:
authority grant Failure Mode: The Benchmark Island A system performs beautifully on a fixed benchmark.
Production gradually changes.
The benchmark remains static.
The agent continues claiming competence based on evidence from a disappearing workload.
This is why competence evidence needs:
production cohorts rolling validation drift detection version binding Benchmarks are evidence.
They are not eternal certification.
Failure Mode: OOD Detector Becomes the New Oracle You build a sophisticated novelty model.
Soon the runtime trusts:
ood_score = 0.17 without understanding where it came from.
Now the safety boundary depends on another opaque model.
Keep deterministic exclusions and hard risk rules outside it.
Use the detector as evidence, not sovereign authority.
Failure Mode: Competence Only Expands Successful cases accumulate.
The envelope grows.
But it never shrinks after:
model drift verifier drift tool changes incidents new workload regimes That creates stale authority.
Envelope contraction must be a normal operation.
Failure Mode: Every OOD Task Goes to a Human That sounds safe.
It can also destroy the system.
Human review capacity is finite.
If every unfamiliar low-risk task becomes an escalation, reviewers become overloaded and rubber-stamping increases.
Use the authority hierarchy.
For example:
OOD + read-only → allow observation OOD + reversible sandbox → allow isolated experiment OOD + consequential write → require human approval OOD + prohibited domain → refuse Competence should contract authority proportionally to risk.
Failure Mode: The Human Approval Expands Competence A human approves one unfamiliar operation.
The system records:
success Then treats the whole task class as validated.
That is not enough.
One approved success is evidence.
It is not broad competence.
Keep sample size, cohort scope, verifier strength, and operating conditions explicit.
Failure Mode: Weak Verification Creates Fake Competence Suppose an agent is evaluated on tasks where success is judged by another similar model.
The benchmark reports 97% success.
Then deterministic checks reveal substantial false success.
The original competence claim was built on a weak measurement system.
Competence evidence is only as strong as its verifier.
This is why Step 23’s verifier-drift work and Step 27’s verifier coverage must feed directly into the envelope.
Failure Mode: Competence Is Treated as a Model Property Teams say:
Model X is competent at coding. But production competence depends on the full system:
model + prompt + tools + retrieval + state quality + router + search policy + verifier + authority controls + operating environment A model swap can change competence.
A verifier outage can change competence.
A tool upgrade can change competence.
A workload shift can change competence.
The competence envelope belongs to the behavioral system, not the foundation model alone.
A Minimal Implementation You do not need a research project to begin.
Start with four things.
1. Cohort labels Record task family, risk class, environment, and verifier profile.
2. Reliability evidence For each cohort, track:
verified success false success UNKNOWN sample size 3. Hard exclusions Encode known unsupported/high-risk regimes.
4. Authority mapping Map competence level to maximum authority.
For example:
AUTHORITY_BY_COMPETENCE = { "VALIDATED": "A2", "SUPPORTED": "A2", "LIMITED": "A1", "UNVALIDATED": "A1", "EXCLUDED": "A0", } Then refine from evidence.
A More Complete Architecture task | task classifier | context fingerprint | +------------+-------------+ | | competence evidence store hard policy | | historical cohorts exclusions incidents risk limits SLO state verifier rules release bindings authority caps | | +------------+-------------+ | OOD evaluator | competence decision | authority/escalation / | \ autonomous human refuse | | | execute review record \ | / verified outcome | evidence update The outcome feeds the evidence store.
But promotion remains governed by release policy.
The runtime does not automatically grant itself more power.
Competence and Expected Value of Information Step 18 asked:
what should the agent observe next? Competence checking adds a useful target:
what evidence would move this task from UNVALIDATED to SUPPORTED? Maybe the missing evidence is:
exact database schema sandbox execution stronger tests current infrastructure topology independent source human domain judgment Now information gathering is tied to authority.
That can make the system dramatically more efficient.
Instead of endlessly “thinking harder,” the agent gathers the specific evidence required to cross a known boundary.
Competence and Dynamic Budgets Step 16 introduced dynamic compute budgets.
An OOD task should not automatically get unlimited compute.
Instead:
novelty high ↓ identify missing competence evidence ↓ spend budget on evidence acquisition ↓ re-evaluate competence ↓ if still unsupported: escalate / defer / refuse This is a better use of compute than generating ten more answers to the same unfamiliar problem.
Competence and Incident Forensics Step 26 can now ask a new question after incidents:
Was the task outside the competence envelope before execution? Possible answers:
YES — OOD detected but authority policy failed YES — OOD detector missed it NO — task was in validated envelope, execution failed UNKNOWN — historical competence evidence incomplete That distinction matters for remediation.
If the task was obviously outside the envelope, improving the model may be the wrong fix.
The fix may be:
better detection stronger authority contraction better escalation better policy enforcement Competence and Reliability Economics Step 28 prioritized reliability investments.
Competence envelopes add another measurable remediation surface.
Suppose many false-success incidents share:
low historical cohort density high environment novelty weak verifier match You may discover that improving competence detection provides more reliability return than upgrading the model.
That is exactly the kind of comparison the series is trying to make possible.
Competence Is a Claim, Not a Feeling This is perhaps the most important idea in the post.
When a system says:
this task is inside the competence envelope that statement should mean:
we have versioned evidence on sufficiently comparable tasks under sufficiently comparable conditions with sufficiently strong verification showing that this authority level meets our reliability contract That is a defensible claim.
It can be audited.
It can be disproved.
It can be updated.
It can shrink.
It can grow.
It is far stronger than:
the model seems confident What to Build First If your production agent currently has no competence-envelope machinery, do not begin with sophisticated density estimation.
Start here:
1. classify tasks into meaningful cohorts 2. attach verified outcomes to those cohorts 3. record verifier strength 4. define hard unsupported regimes 5. map cohort evidence to maximum authority 6. return UNVALIDATED when evidence is absent 7. require human approval for consequential OOD actions 8. track false in-distribution and false OOD decisions That already gives you a large improvement over confidence-based autonomy.
Then add learned novelty detection only where the simpler system demonstrably misses important cases.
The Architecture So Far The advanced-agent stack now looks less like a clever prompt loop and more like a production control system:
models / tools / memory / search ↓ trajectory observability ↓ controlled adaptation ↓ policy optimization ↓ dynamic budgeting ↓ typed uncertainty ↓ value of information ↓ speculative execution ↓ distributed coordination ↓ platform scheduling ↓ failure containment ↓ behavioral drift detection ↓ release engineering ↓ replay + provenance ↓ incident forensics ↓ SLOs + error budgets ↓ reliability prioritization ↓ authority boundaries ↓ competence envelopes Notice what happened.
The model moved further and further away from being the sole authority.
That is not accidental.
A production agent becomes more trustworthy not because the model becomes omniscient, but because the system becomes better at knowing:
what it knows what it can verify where it has evidence where it does not what it may do when it must stop The Next Problem: Safe Exploration Outside the Envelope Competence envelopes create an immediate next question.
If a task is outside the validated envelope, do we simply stop forever?
No.
A useful system needs a disciplined way to explore new territory without granting production authority prematurely.
That means building sandboxed capability acquisition:
OOD task ↓ restricted sandbox ↓ experiments / probes / simulation ↓ strong verification ↓ new evidence ↓ offline competence update proposal ↓ shadow / canary ↓ possible envelope expansion The key idea will be:
Learn outside the authority boundary before expanding the authority boundary.
That is where we go next.