Advanced Agents From First Principles 01: Does Your AI Agent Fail on Complex Reasoning Tasks? Treat Chain of Thought as Computation, Not Proof

Most developers first encounter chain of thought as a prompting trick:

Think step by step.

That framing is too shallow for agent engineering.

For an advanced agent, the useful idea is not that the model should produce a long explanation. The useful idea is that a difficult task may benefit from intermediate computational state before the system commits to an action or answer.

That is a very different claim.

A reasoning trace can help a system decompose a problem, preserve intermediate conclusions, identify missing information, decide what to verify next, and expose places where search or tools should be used.

But none of that makes the trace true.

The central rule of this post is:

Chain of thought is computation, not proof.

A plausible reasoning trace is still model output. It can contain incorrect assumptions, invented facts, invalid arithmetic, bad causal links, or a perfectly coherent path to the wrong answer.

So the engineering question is not:

How do I make the model explain itself more?

It is:

Where does intermediate computation improve verified task success, and how should the runtime use that computation without mistaking it for evidence?

This post builds that distinction from first principles.

The Search Problem: “Why Does My AI Agent Fail on Complex Reasoning Tasks?”

A common pattern looks like this:

simple task
    ↓
model succeeds

complex task
    ↓
model jumps directly to answer
    ↓
important constraint omitted
    ↓
wrong answer

The failure is often described as “the model cannot reason.”

Sometimes that is true.

But often the problem is more specific: the runtime asks the model to collapse too much computation into one opaque output.

Consider a coding agent asked to repair a failing test suite.

A one-shot path may look like:

issue description
      ↓
model
      ↓
patch

A more useful computational path might be:

issue description
      ↓
identify failing behavior
      ↓
inspect relevant code
      ↓
form hypotheses
      ↓
select next diagnostic
      ↓
observe result
      ↓
revise hypothesis
      ↓
propose patch
      ↓
run tests

The improvement does not come from making the prose longer.

It comes from creating intermediate decisions that can interact with evidence.

One-Shot Generation Compresses Too Much

The simplest agent asks a model to go directly from input to output:

def solve(task: str, model) -> str:
    return model(task)

That is often the correct architecture.

Do not add a reasoning mechanism merely because the task looks intellectual.

But for some tasks, direct generation forces the model to implicitly perform several operations at once:

understand task
    +
identify constraints
    +
retrieve relevant facts
    +
choose method
    +
perform calculation
    +
check result
    +
format answer

If any hidden sub-operation fails, the final answer can fail without giving the runtime a useful point at which to intervene.

Intermediate computation changes the shape:

input
 ↓
intermediate state 1
 ↓
intermediate state 2
 ↓
verification / tool use
 ↓
intermediate state 3
 ↓
answer

Now the runtime has places where it can inspect, verify, branch, retry, or gather more information.

Chain of Thought Is Not Necessarily Visible Prose

This distinction matters enormously.

Developers often equate chain of thought with a long text field containing sentences such as:

First I will consider...
Then I will calculate...
Therefore...

That is only one representation.

For agent systems, intermediate computation can be represented as structured state instead.

For example:

from dataclasses import dataclass, field


@dataclass
class ReasoningState:
    goal: str
    known_facts: list[str] = field(default_factory=list)
    unknowns: list[str] = field(default_factory=list)
    hypotheses: list[str] = field(default_factory=list)
    next_checks: list[str] = field(default_factory=list)
    constraints: list[str] = field(default_factory=list)

That is often more useful than unrestricted prose.

Why?

Because the runtime can validate and operate on it.

free-form reasoning
      ↓
hard to inspect mechanically

structured reasoning state
      ↓
known fields
      ↓
validation
      ↓
tool routing
      ↓
search / verification

This is a recurring theme in agent engineering:

When intermediate reasoning affects control flow, represent the parts that matter as data.

A Minimal Reasoning Pipeline

Suppose we want the model to decompose a difficult task before answering.

We can separate decomposition from execution.

from dataclasses import dataclass


@dataclass
class ReasoningPlan:
    interpretation: str
    constraints: list[str]
    steps: list[str]
    verification: list[str]


def reason(task: str, model) -> ReasoningPlan:
    raw = model({
        "task": task,
        "instruction": (
            "Return a structured plan containing an interpretation, "
            "constraints, steps, and verification checks."
        ),
    })
    return parse_reasoning_plan(raw)

Then execution can be separated:

def execute_reasoning_plan(plan: ReasoningPlan, runtime):
    observations = []

    for step in plan.steps:
        observation = runtime.execute(step)
        observations.append(observation)

    return observations

And success can be evaluated separately again:

def verify(plan: ReasoningPlan, observations, verifier):
    return verifier.check(
        constraints=plan.constraints,
        verification=plan.verification,
        observations=observations,
    )

The key architecture is:

reasoning
   ≠
execution
   ≠
verification

That separation is far more useful than merely asking for “more reasoning.”

Why Long Reasoning Can Still Be Wrong

Consider this hypothetical trace:

The service is failing because the database connection pool is exhausted.
The latency spike supports this.
Therefore increasing the pool size should fix the incident.

It is coherent.

It may even sound expert.

But perhaps the actual problem is a downstream dependency timing out, causing connections to remain occupied longer.

Increasing the pool size might simply delay the failure.

The reasoning trace contains a hypothesis:

pool exhaustion is root cause

That is not proof.

A better runtime turns the hypothesis into a test:

hypothesis
    ↓
what observation would support/refute it?
    ↓
inspect pool saturation
    ↓
inspect connection duration
    ↓
inspect downstream latency
    ↓
update belief

Now chain of thought is being used as a generator of testable intermediate state.

That is much stronger.

Separate Claims From Evidence

A useful reasoning representation can distinguish claims from evidence.

from dataclasses import dataclass, field


@dataclass
class Claim:
    text: str
    evidence_ids: list[str] = field(default_factory=list)
    status: str = "unverified"

Then the runtime can enforce a simple rule:

def can_promote(claim: Claim, evidence_store) -> bool:
    return all(
        evidence_store.exists(evidence_id)
        for evidence_id in claim.evidence_ids
    )

The model may say:

The authentication middleware is dropping the header.

The runtime records:

CLAIM: authentication middleware drops header
STATUS: unverified

Then tools gather evidence:

request trace
middleware logs
integration test

Only then does the status change.

This prevents a dangerous collapse:

model thought it
      ↓
therefore system believes it

The Reasoning Scratchpad and the Control Plane Are Different

Another important distinction is between temporary reasoning state and durable runtime state.

A scratchpad might contain:

Maybe file A is relevant.
Could also be file B.
Need to inspect the parser first.

That is useful temporary computation.

But it should not automatically become durable memory.

Compare:

scratchpad
    ↓
provisional hypotheses
    ↓
throw away freely

with:

runtime state
    ↓
verified files inspected
commands executed
artifacts produced
remaining tasks

The latter needs deterministic persistence.

The previous memory post made this distinction for long-term memory. The same principle applies here:

Do not turn provisional reasoning into system truth merely because it appeared earlier in the trajectory.

Reasoning as a State Machine

A more disciplined design is to represent reasoning stages explicitly.

from enum import Enum


class ReasoningPhase(str, Enum):
    UNDERSTAND = "understand"
    DECOMPOSE = "decompose"
    INVESTIGATE = "investigate"
    PROPOSE = "propose"
    VERIFY = "verify"
    COMPLETE = "complete"

Now the system can control transitions:

UNDERSTAND
    ↓
DECOMPOSE
    ↓
INVESTIGATE
    ↓
PROPOSE
    ↓
VERIFY
    ↓
COMPLETE

But transitions should not be model wishful thinking.

For example:

def can_enter_verify(state) -> bool:
    return bool(state.proposed_solution)


def can_complete(state) -> bool:
    return state.verification_status == "PASS"

This turns a vague reasoning process into a runtime protocol.

Use Tools to Break Reasoning Out of the Model

Suppose an agent needs to answer:

Which function introduced the regression?

The weakest architecture is:

repository text
     ↓
LLM guesses

A stronger reasoning architecture is:

form hypothesis
     ↓
git diff
     ↓
inspect call sites
     ↓
run targeted test
     ↓
inspect failure
     ↓
update hypothesis

The computation is distributed across:

model
+
tools
+
environment
+
verification

This is one of the most important advanced-agent ideas:

The model does not need to internally simulate work that the environment can perform more reliably.

Use computation where it is strongest.

Arithmetic belongs in a calculator.

Repository history belongs in Git.

Database state belongs in the database.

Test outcomes belong in the test runner.

The model’s intermediate computation should help decide which external computation to request next.

Chain of Thought as Tool Selection

This gives us a useful interpretation of reasoning:

current evidence
      ↓
what uncertainty matters most?
      ↓
which observation would reduce it?
      ↓
select tool
      ↓
observe
      ↓
update state

That is much closer to an agent than a long essay about what the model thinks.

For example, a debugging agent might hold:

state.unknowns = [
    "whether parser receives malformed input",
    "whether failure occurs before persistence",
]

The model chooses which unknown to reduce first.

next_check = {
    "tool": "inspect_logs",
    "arguments": {
        "component": "parser",
        "request_id": "abc123",
    },
}

Now reasoning directly guides evidence acquisition.

What Should Be Structured?

Not everything needs a schema.

But fields that influence control flow usually benefit from structure.

Useful candidates include:

goal
constraints
unknowns
hypotheses
candidate actions
required evidence
verification checks
confidence / uncertainty
termination reason

For example:

@dataclass
class InvestigationState:
    goal: str
    constraints: list[str]
    hypotheses: list[str]
    unresolved_questions: list[str]
    evidence: list[str]
    next_action: dict | None

This allows the runtime to answer questions that free-form prose obscures:

How many unresolved questions remain?
Which hypothesis has no evidence?
Did the agent already test this idea?
Which constraint has not been checked?
Why did the agent choose this tool?

The Common Failure: Verbosity Masquerading as Reasoning

One of the most common mistakes is to increase reasoning-token usage and assume capability improved.

You may see:

short answer: 300 tokens
reasoning answer: 4,000 tokens

and conclude:

more tokens
   ↓
more thought
   ↓
better reasoning

That inference is not safe.

Longer traces can contain:

• repetition
• rationalization
• irrelevant decomposition
• invented details
• premature assumptions
• circular reasoning
• duplicated checks
• post-hoc explanations

The right metric is not reasoning length.

It is verified task performance.

A useful experiment is:

baseline: direct answer
variant A: structured decomposition
variant B: decomposition + tools
variant C: decomposition + tools + verification

Measure:

verified success
latency
model calls
tool calls
tokens
cost per successful task

Then keep the simplest architecture that wins.

The Common Failure: Self-Consistency of One Bad Assumption

A model can construct a long, internally coherent trace around a bad premise.

For example:

Assumption: request reaches service B.

Step 1: service B parses request.
Step 2: service B writes database row.
Step 3: database constraint fails.

If the request never reached service B, every later step is irrelevant.

This motivates assumption checkpoints.

@dataclass
class Assumption:
    statement: str
    must_verify: bool
    evidence_id: str | None = None

Before expensive downstream reasoning, the runtime can check critical assumptions.

assumption
   ↓
cheap verification available?
   ↓ yes
verify first
   ↓
continue only if supported

This often saves both tokens and wrong work.

The Common Failure: Reasoning After the Answer

Another subtle failure occurs when the model effectively decides on an answer first and then generates a plausible rationale.

For agent engineering, the remedy is not philosophical.

It is architectural.

Make intermediate decisions causally matter.

Instead of:

model produces answer + explanation

use:

model proposes next check
      ↓
runtime executes it
      ↓
observation enters state
      ↓
model receives observation
      ↓
next decision

The external observation constrains the next step.

That makes the reasoning trajectory operational rather than merely descriptive.

Reasoning Checkpoints

Long tasks benefit from checkpoints where the runtime asks whether the trajectory is still valid.

A checkpoint might inspect:

Are the original constraints still represented?
Have any assumptions been falsified?
Has new evidence changed the problem?
Are we repeating the same hypothesis?
Is the current branch making progress?
Can any remaining unknown be verified directly?

A simple checkpoint object:

@dataclass
class ReasoningCheckpoint:
    constraints_satisfied: bool
    assumptions_supported: bool
    progress_detected: bool
    unresolved_questions: list[str]
    recommended_action: str

Possible recommendations:

CONTINUE
GATHER_EVIDENCE
REVISE_HYPOTHESIS
BACKTRACK
VERIFY
STOP

Notice how this begins to connect chain of thought to later advanced techniques such as search and MCTS.

Reasoning vs Search

Chain of thought and search are related but different.

A single reasoning trajectory looks like:

A
 ↓
B
 ↓
C
 ↓
D

Search preserves alternatives:

        A
      /   \
     B     C
    / \   / \
   D   E F   G

Chain of thought can improve a single path.

Search asks whether we should maintain multiple paths.

The escalation rule is:

Does one structured reasoning trajectory work?
        ↓ yes
stop there

        ↓ no
Are early decisions uncertain and consequential?
        ↓ yes
preserve alternatives with search

Do not build a tree merely because you can.

Reasoning vs Best-of-N

Best-of-N generates several complete candidates:

prompt
 ├→ answer A
 ├→ answer B
 ├→ answer C
 └→ answer D
        ↓
      select

Reasoning decomposes one trajectory:

prompt
 ↓
state 1
 ↓
state 2
 ↓
state 3
 ↓
answer

Use Best-of-N when diversity among complete outputs helps.

Use intermediate reasoning when later decisions should depend on intermediate observations.

You can combine them:

multiple reasoning strategies
        ↓
execute / verify
        ↓
select best supported result

But again, every extra path costs inference.

Reasoning vs Critique and Revision

Critique/revision operates after a candidate exists:

candidate
   ↓
critique
   ↓
revision

Intermediate reasoning operates before or during construction:

problem
   ↓
decompose
   ↓
investigate
   ↓
construct

The distinction matters because some failures are easier to prevent than repair.

If the agent misses a fundamental constraint at the beginning, repeated rewriting may never recover efficiently.

A reasoning checkpoint can surface the constraint before generation commits too far.

Reasoning and Verification Must Remain Separate

Suppose the reasoning state contains:

All required tests should now pass.

That is still only a prediction.

The verifier must run:

pytest

and return:

PASS
FAIL
UNKNOWN

The correct architecture is:

reasoning predicts
      ↓
environment tests
      ↓
verification decides

Not:

reasoning predicts
      ↓
reasoning confirms itself

This separation is one of the main reasons the verification post closed the core series before we entered advanced orchestration.

A Complete Small Reasoning Agent

Here is a simplified architecture that keeps the pieces separate.

from dataclasses import dataclass, field
from typing import Protocol


class Model(Protocol):
    def __call__(self, payload: dict) -> dict:
        ...


@dataclass
class Evidence:
    id: str
    source: str
    content: str


@dataclass
class ReasoningState:
    goal: str
    constraints: list[str] = field(default_factory=list)
    hypotheses: list[str] = field(default_factory=list)
    unknowns: list[str] = field(default_factory=list)
    evidence: list[Evidence] = field(default_factory=list)
    proposed_answer: str | None = None


class ReasoningAgent:
    def __init__(self, model: Model, tools, verifier):
        self.model = model
        self.tools = tools
        self.verifier = verifier

    def run(self, goal: str, max_steps: int = 8):
        state = ReasoningState(goal=goal)

        for step in range(max_steps):
            decision = self.model({
                "goal": state.goal,
                "constraints": state.constraints,
                "hypotheses": state.hypotheses,
                "unknowns": state.unknowns,
                "evidence": [e.content for e in state.evidence],
                "instruction": (
                    "Choose one action: gather_evidence, update_state, "
                    "propose_answer, or request_verification."
                ),
            })

            action = decision["action"]

            if action == "gather_evidence":
                tool_name = decision["tool"]
                arguments = decision["arguments"]
                result = self.tools.execute(tool_name, arguments)
                state.evidence.append(
                    Evidence(
                        id=f"evidence-{step}",
                        source=tool_name,
                        content=result,
                    )
                )
                continue

            if action == "update_state":
                state.constraints = decision.get(
                    "constraints", state.constraints
                )
                state.hypotheses = decision.get(
                    "hypotheses", state.hypotheses
                )
                state.unknowns = decision.get(
                    "unknowns", state.unknowns
                )
                continue

            if action == "propose_answer":
                state.proposed_answer = decision["answer"]
                continue

            if action == "request_verification":
                if state.proposed_answer is None:
                    continue

                result = self.verifier.verify(
                    goal=state.goal,
                    answer=state.proposed_answer,
                    evidence=state.evidence,
                )

                if result.status == "PASS":
                    return state.proposed_answer

                if result.status == "FAIL":
                    state.unknowns.extend(result.failures)
                    state.proposed_answer = None
                    continue

        raise RuntimeError("Reasoning budget exhausted")

This is still intentionally small.

But notice what has changed compared with a prompt that says “think step by step.”

The runtime now owns:

• step budget
• tool execution
• evidence records
• verification
• completion

The model proposes intermediate computation.

The runtime controls the process.

Application: Coding Agents

Coding agents are an excellent example because the environment provides strong evidence.

A weak reasoning loop:

read issue
 ↓
think about code
 ↓
write patch
 ↓
claim fixed

A stronger one:

read issue
 ↓
identify expected behavior
 ↓
locate relevant code
 ↓
form failure hypothesis
 ↓
run targeted diagnostic
 ↓
update hypothesis
 ↓
edit isolated branch/worktree
 ↓
run targeted tests
 ↓
run regression checks
 ↓
inspect diff

Intermediate reasoning is useful for:

• choosing what file to inspect
• choosing which test to run
• identifying likely invariants
• deciding whether failure evidence supports a hypothesis
• deciding when to broaden search

But Git, tests, type checkers, linters, and runtime behavior remain stronger evidence than prose reasoning.

Application: Research Agents

A research agent often fails because it jumps from query to conclusion too quickly.

Useful intermediate state includes:

question
subquestions
candidate claims
missing evidence
source quality
contradictions
unresolved uncertainty

A stronger research path:

question
 ↓
decompose claims
 ↓
search primary sources
 ↓
extract evidence
 ↓
identify contradictions
 ↓
search missing evidence
 ↓
write synthesis
 ↓
check every material claim has support

The reasoning trace helps organize the investigation.

The sources provide the evidence.

That distinction protects the system from citing its own generated explanation as if it were research.

Application: Debugging and Incident Response

Incident response is where unverified reasoning can become expensive.

A model might quickly conclude:

CPU saturation caused the outage.

A disciplined agent instead records:

hypothesis: CPU saturation is causal

needed evidence:
- CPU timing relative to errors
- request latency
- queue depth
- dependency latency

Then it collects observations.

This is essentially scientific debugging:

hypothesis
 ↓
prediction
 ↓
measurement
 ↓
update

Chain of thought becomes a mechanism for choosing informative experiments.

Application: Data and Analytics Agents

Suppose an analytics agent must answer:

Why did conversion fall yesterday?

A bad one-shot answer might invent a plausible narrative.

A useful reasoning state might be:

metric definition
comparison window
possible segmentation changes
tracking changes
traffic-source mix
funnel-stage deltas
known incidents

Then the agent queries actual data.

reasoning
 ↓
SQL / metrics tools
 ↓
observations
 ↓
updated hypothesis
 ↓
verified explanation

The model helps coordinate analysis.

The data determines what happened.

Application: Browser Agents

Browser tasks require reasoning about hidden state and sequential constraints.

For example:

goal: submit application

Intermediate state may include:

required fields
completed fields
missing documents
current page
validation errors
submission status

The reasoning mechanism decides what to do next.

But successful submission should be verified from the page/application state—not from a generated sentence saying “the form has been submitted.”

Application: Planning and Scheduling Agents

Planning agents benefit when constraints must be combined before committing.

Useful intermediate computation:

hard constraints
soft preferences
resource availability
dependencies
conflicts
candidate schedules

Then deterministic solvers or constraint checks can test candidate plans.

This is an important pattern:

LLM proposes structure
      ↓
solver checks feasibility
      ↓
LLM interprets / revises

Do not ask a language model to replace a deterministic constraint solver when one exists.

Application: Customer Support Agents

A support agent may need to reason about:

customer intent
account state
order state
policy constraints
possible resolution
escalation criteria

But customer-specific facts must come from account/order systems.

A strong architecture is:

interpret request
 ↓
identify required facts
 ↓
query systems
 ↓
apply policy
 ↓
propose resolution
 ↓
validate permissions
 ↓
execute / escalate

The reasoning is useful coordination.

The business systems remain authoritative.

Application Matrix

The recurring architecture is:

model computation
      ↓
external observation
      ↓
state update
      ↓
model computation

not:

model computation
      ↓
more model computation
      ↓
model agrees with itself

When Chain of Thought Makes Things Worse

Intermediate reasoning adds cost and new failure modes.

It can hurt when:

• the task is already trivial
• the model over-decomposes
• reasoning anchors on an early mistake
• intermediate state bloats context
• repeated reflection adds no evidence
• the runtime stores provisional thoughts as facts
• reasoning latency dominates the task
• private or sensitive intermediate data is retained unnecessarily
• the final answer quality does not improve

A good baseline experiment is mandatory.

A: direct generation
B: structured reasoning
C: structured reasoning + tools
D: structured reasoning + tools + verification

Do not compare only average answer scores.

Measure operational outcomes.

Metrics That Actually Matter

For reasoning agents, useful metrics include:

Outcome

verified task success

Efficiency

model calls
tokens
latency
tool calls
cost per verified success

Reasoning usefulness

fraction of intermediate checks that change the trajectory
fraction of hypotheses tested
assumption failure rate
repeated-hypothesis rate
unverified-claim rate

Verification

false-pass rate
false-fail rate
UNKNOWN rate

Progress

unknowns reduced per step
constraints resolved per step
useful evidence acquired per call

A reasoning architecture that produces twice as many tokens but no measurable improvement is not more advanced.

It is simply more expensive.

A Useful Diagnostic: Did the Reasoning Change Anything?

This question is surprisingly powerful.

For every intermediate reasoning step, ask:

Did it change:
- the next action?
- the selected tool?
- the active hypothesis?
- the candidate solution?
- the verification plan?

If not, the step may be decorative.

You can log this directly:

@dataclass
class ReasoningEvent:
    event_type: str
    content: str
    changed_control_flow: bool

Then measure:

control-flow-changing reasoning events
--------------------------------------
all reasoning events

If that number is tiny, the system may be generating explanations rather than useful computation.

Another Diagnostic: What Could Falsify This Step?

For a material hypothesis, ask:

What observation would prove this wrong?

If the system cannot name one, the hypothesis may be too vague to guide action.

For example:

"Something is wrong with caching"

is weak.

Better:

"The cache key omits tenant_id, causing cross-tenant reuse."

Now we can inspect:

key construction
cache entries
request tenant IDs

Reasoning becomes operational.

Confidence Is Not Evidence Either

Models may attach confidence to intermediate conclusions.

That can help prioritize checks, but confidence should not replace verification.

high confidence
     ≠
correct

A better use is compute allocation:

low uncertainty
   ↓
cheap path

high uncertainty
   ↓
more evidence / search / specialist

This is where chain of thought connects directly to adaptive agents later in the series.

Reasoning Traces and Privacy

Intermediate reasoning can contain more information than the final answer.

Production systems should decide deliberately:

• what reasoning state is retained
• what is logged
• what is sent to external services
• what is visible to users
• what contains secrets or personal data
• what should expire

A useful architecture separates:

ephemeral computation
      ↓
minimum durable trace

You often need provenance like:

tool called
input hash
observation ID
decision selected
verification result

without retaining every free-form intermediate token forever.

Reasoning Trace vs Decision Trace

For production agents, a decision trace is often more useful than a full free-form reasoning transcript.

Example:

{
  "step": 4,
  "state_hash": "...",
  "decision": "run_targeted_test",
  "reason": "hypothesis H2 lacks runtime evidence",
  "tool": "pytest",
  "result_id": "obs-421"
}

This captures the information needed for debugging:

what did the agent believe mattered?
what did it do?
what evidence came back?

without making the entire system dependent on prose traces.

Treat Reasoning as a Budget

Reasoning consumes compute.

So give it a budget.

@dataclass
class ReasoningBudget:
    max_steps: int
    max_model_calls: int
    max_tool_calls: int
    max_tokens: int

Then escalate only when needed.

simple task
   ↓
direct path

failure / uncertainty
   ↓
structured reasoning

still unresolved
   ↓
search / specialist / stronger model

This is much better than enabling maximum reasoning for every request.

Adaptive Reasoning Depth

One advanced extension is to vary reasoning effort based on the task.

For example:

def reasoning_budget(task, classifier):
    difficulty = classifier(task)

    if difficulty == "low":
        return 1

    if difficulty == "medium":
        return 4

    return 10

But difficulty estimation itself can fail.

So runtime evidence should also trigger escalation:

direct answer fails verification
       ↓
reason more

reasoning branch remains uncertain
       ↓
search alternatives

search cannot discriminate
       ↓
stronger verifier / specialist

This is the beginning of adaptive compute allocation.

The Evidence-First Rule

The Advanced Agents series keeps the same rule as the previous series:

Do not add a mechanism because it sounds more intelligent. Add it because you can name the failure it is supposed to fix and measure whether it fixes it.

For chain of thought, the hypothesis might be:

Complex tasks fail because important constraints are lost before action selection.

Then test:

baseline
    direct generation

variant
    structured constraints + intermediate checks

Measure:

constraint violation rate
verified success
latency
cost

If constraints are still violated at the same rate, the mechanism did not solve the claimed problem.

An Experiment Template

from dataclasses import dataclass


@dataclass
class ReasoningExperimentResult:
    architecture: str
    verified_success_rate: float
    constraint_failure_rate: float
    avg_model_calls: float
    avg_tool_calls: float
    avg_latency_ms: float
    avg_tokens: float
    cost_per_success: float

Compare:

direct
structured reasoning
reasoning + tools
reasoning + tools + verification

Then ask:

What did the extra computation buy us?

That question should follow every advanced-agent mechanism in this series.

What This Unlocks Next

Once intermediate computation is explicit, we can do something more interesting.

Instead of generating one reasoning trajectory:

A → B → C → D

we can generate several:

A → B → C
A → E → F
A → G → H

Then compare where they agree.

That leads to the next advanced technique:

self-consistency

But self-consistency introduces another trap:

many agents / samples agree
      ≠
answer is true

Agreement can come from correlated model errors.

So the same evidence-first boundary remains.

Final Architecture

The useful form of chain of thought for an agent looks less like this:

Please explain your reasoning in detail.

and more like this:

goal
 ↓
structured intermediate state
 ↓
identify uncertainty
 ↓
choose evidence-producing action
 ↓
observe environment
 ↓
update hypotheses
 ↓
propose result
 ↓
external verification

The model provides flexible computation.

The runtime supplies structure.

Tools supply observations.

Verification supplies evidence.

That is the important transition from prompting to agent architecture.

And it gives us the rule to carry into every later post:

Reasoning can tell the agent what to investigate next. Only evidence can tell the system what actually happened.

Next: Self-Consistency

The next post will ask another common question:

Why does my reasoning agent give a different answer every time?

We will build self-consistency from first principles: sample multiple reasoning trajectories, measure agreement, separate consensus from correctness, detect correlated failures, and decide when disagreement should trigger search, verification, or specialist escalation instead of a majority vote.