Advanced Agents From First PrinciplesStage 04
Behavioral Production Engineering · Steps 23–26Chapter 25 of 45

Can You Reproduce an Agent Run Months Later? Add Deterministic Replay and Provenance

Page content

A production agent fails on Tuesday.

You inspect it on Wednesday and cannot reproduce the failure.

Three weeks later someone asks a harder question:

What exactly did this agent see, which release was running, which tools were called, which evidence was used, and why did the verifier accept the result?

If the answer is “we have some logs”, you do not yet have reproducibility.

You have fragments.

Advanced agents accumulate hidden variability quickly:

  • model versions change,
  • prompts change,
  • retrieval indexes change,
  • tools return different state,
  • schedulers make different decisions,
  • search branches complete in different orders,
  • external APIs mutate,
  • memory evolves,
  • policies are promoted and rolled back,
  • verifiers change,
  • and distributed workers may execute different attempts of the same logical operation.

Step 24 treated the full behavioral surface as a release artifact.

Now we need to make every important run reconstructable.

The core rule is:

A run is reproducible only when its behavior can be tied to immutable inputs, versioned policy, external observations, side effects, and verification evidence.

That does not mean every model token must be perfectly deterministic.

It means the system should preserve enough evidence to distinguish:

same release + same recorded observations
            ↓
logical replay

same release + live dependencies
            ↓
re-execution

new release + old recorded observations
            ↓
counterfactual replay

Those are different experiments.

Treating them as equivalent is one of the easiest ways to fool yourself while debugging agent systems.

1. Replay Is Not One Thing

The word replay is overloaded.

A useful runtime distinguishes at least three modes.

Exact artifact replay

Reconstruct the original run from recorded artifacts without calling mutable external dependencies again.

Examples:

  • reuse the exact prompt payload,
  • reuse recorded tool responses,
  • reuse the exact retrieved documents,
  • reuse the exact verifier inputs,
  • replay the recorded decision graph.

This is the closest thing to a forensic reconstruction.

Logical replay

Run the same behavioral release again against controlled or recorded dependencies.

The model may still produce a different sampled output, but the environment and policy surface are constrained enough that differences are meaningful.

Counterfactual replay

Take an old trajectory and ask:

What would release B have done if it had received the same evidence release A received?

This is crucial for offline policy evaluation, release comparison, routing calibration, and regression diagnosis.

The distinction matters because a live rerun against today’s web, today’s repository state, today’s model alias, and today’s retrieval index is not a replay of last month’s incident.

It is a new run.

2. The Replay Manifest

Every important run should be bound to an immutable replay manifest.

A minimal structure might look like this:

from dataclasses import dataclass
from typing import Mapping, Sequence


@dataclass(frozen=True)
class ReplayManifest:
    run_id: str
    release_id: str
    task_hash: str
    environment_snapshot_id: str
    tool_registry_version: str
    retrieval_snapshot_id: str
    memory_snapshot_id: str
    scheduler_policy_version: str
    verifier_version: str
    observation_ids: Sequence[str]
    side_effect_ids: Sequence[str]
    artifact_hashes: Mapping[str, str]

The point is not the dataclass.

The point is that the run is no longer defined by a vague timestamp plus logs.

It is defined by a set of references whose content can be checked.

3. Version Names Are Not Enough

Suppose the log says:

model = gpt-x
prompt = repair-v4

That is weak provenance.

A model alias may later point to different serving weights.

A prompt named repair-v4 may have been edited in place.

A tool named repo_search may now use a different implementation.

Prefer immutable identity:

model_provider
model_family
provider_model_id
serving_revision_if_available
prompt_sha256
tool_schema_hash
tool_implementation_commit
policy_hash
verifier_hash
retrieval_snapshot_hash

If something cannot be immutably identified, record that limitation explicitly.

Do not pretend a mutable alias is a stable artifact.

4. Hash the Inputs That Matter

A replay system should be able to answer:

Are these bytes the same bytes the original run used?

For text artifacts, structured payloads, prompts, retrieved documents, configuration, tool responses, and verifier inputs, content hashes are extremely useful.

For example:

import hashlib
import json


def canonical_hash(payload: dict) -> str:
    encoded = json.dumps(
        payload,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False,
    ).encode("utf-8")
    return hashlib.sha256(encoded).hexdigest()

The canonicalization rule must itself be stable and versioned.

Otherwise two semantically equivalent dictionaries can hash differently for irrelevant formatting reasons.

5. Record Observations, Not Just Tool Calls

This is not enough:

tool = git_status
status = success

You need the observation that changed the agent’s state.

For example:

{
  "observation_id": "obs-88",
  "tool": "git_status",
  "tool_version": "7f4d...",
  "input_hash": "...",
  "output_hash": "...",
  "observed_at": "2026-08-09T11:00:00Z",
  "environment_snapshot": "repo-4c91...",
  "freshness": "live",
  "authority": "repository",
  "scope": "worktree:/workspace/a"
}

Step 17 separated different uncertainty types.

Replay requires preserving the evidence used to reduce those uncertainties.

6. Environment State Is Part of the Input

For coding agents, the repository state is not background context.

It is part of the input.

Record at least:

  • repository remote identity,
  • commit SHA,
  • branch/ref,
  • dirty working-tree state if relevant,
  • untracked files if relevant,
  • dependency lockfiles,
  • runtime/container image,
  • environment variables by safe fingerprint,
  • toolchain versions,
  • database migration state when relevant.

For browser agents:

  • page URL,
  • response body or snapshot when permitted,
  • page version/fingerprint,
  • authenticated account scope,
  • locale,
  • important cookies/session identity by safe reference,
  • DOM/screenshot artifact IDs.

For data agents:

  • dataset snapshot,
  • schema version,
  • query text,
  • database transaction/snapshot identity where possible.

For DevOps agents:

  • cluster/account/environment,
  • deployment revision,
  • observed resource versions,
  • change request identity,
  • relevant control-plane state.

Without this, “same task” is often not the same experiment.

7. Separate External Observations From Model Inference

A model saying:

The deployment probably failed because the image tag is stale.

is not the same thing as:

kubectl observed image tag = api:2026-08-08
expected image tag = api:2026-08-09

Replay artifacts should preserve this distinction.

A useful provenance graph might contain nodes such as:

TASK
OBSERVATION
MODEL_OUTPUT
POLICY_DECISION
TOOL_ACTION
SIDE_EFFECT
VERIFIER_EVIDENCE
OUTCOME

with edges such as:

DERIVED_FROM
OBSERVED_BY
SELECTED_BY
CAUSED
VERIFIED_BY
REPLACED
CANCELLED

That lets you reconstruct not just what happened, but what evidence each decision depended on.

8. Preserve the Decision Graph

Step 13 introduced trajectory observability.

For replay, keep the lineage.

A flat timestamp log loses important structure.

Suppose an agent explored three repair branches:

root
├── branch A
│   └── test fail
├── branch B
│   └── verifier PASS
└── branch C
    └── cancelled

You should know:

  • which state each branch started from,
  • which observations each branch received,
  • which scorer or policy ranked it,
  • why branch C was cancelled,
  • whether B’s PASS referred to the exact candidate that was later committed.

The selected result must remain linked to the exact evidence that justified selection.

9. Replay Must Include Scheduler Decisions

Advanced agents do not only generate outputs.

They allocate resources.

Steps 15 through 18 introduced:

  • routing policy,
  • search policy,
  • escalation policy,
  • stopping policy,
  • budget scheduling,
  • typed uncertainty,
  • Expected Value of Information.

Those decisions affect the trajectory.

Record them.

For example:

{
  "decision_id": "decision-41",
  "policy_version": "budget-17",
  "state_hash": "...",
  "alternatives": [
    "run_tests",
    "generate_candidate",
    "escalate_model"
  ],
  "selected": "run_tests",
  "expected_cost": 0.04,
  "expected_value": 0.62,
  "reason_code": "HIGH_VERIFICATION_UNCERTAINTY"
}

You do not need hidden chain-of-thought.

You need the operational decision boundary.

10. Record Side Effects as First-Class Artifacts

A replay system that only records reasoning but not mutations is incomplete.

For every consequential side effect, capture:

operation_id
attempt_id
worker_id
lease_epoch
idempotency_key
precondition evidence
request fingerprint
response fingerprint
postcondition evidence
commit timestamp

Step 20 introduced leases, fencing, retries, and idempotency.

Replay should preserve those identities so you can answer:

Did the same logical operation execute twice?

and:

Was the final state produced by the selected worker or a stale worker?

11. External APIs Break Perfect Replay

Some dependencies are inherently mutable.

Examples:

  • web pages,
  • SaaS APIs,
  • market data,
  • live infrastructure,
  • hosted model endpoints,
  • third-party search engines.

You have two choices:

  1. record the observation when it happened,
  2. call the live dependency again later.

Those are different modes.

For forensic replay, prefer recorded observations whenever policy, privacy, and storage constraints permit.

For live revalidation, call the dependency again but mark the result as new evidence.

Do not overwrite the original artifact.

12. Deterministic Replay Does Not Mean Deterministic Models

This distinction is crucial.

Even with a seed, hosted model execution may not be bit-for-bit reproducible.

Serving infrastructure can change.

Numerical kernels can change.

Parallelism can change.

Sampling implementations can change.

A good replay system therefore does not promise:

same input -> identical tokens forever

unless you actually control the full model runtime and have tested that claim.

Instead, distinguish:

artifact deterministic
policy deterministic
external observation deterministic
model generation nondeterministic

Then define what counts as an acceptable replay.

For example:

  • same route,
  • same required tools,
  • same invariants,
  • same verifier outcome,
  • same accepted behavioral contract,
  • similar cost envelope.

That is often more useful than token identity anyway.

13. Replaying Model Calls

For each model call, preserve:

  • provider,
  • model identifier,
  • revision if available,
  • temperature,
  • top-p,
  • seed if used,
  • max output tokens,
  • structured-output schema,
  • tool schema,
  • system prompt hash,
  • user prompt hash,
  • message payload hash,
  • response payload hash,
  • usage metadata,
  • latency,
  • retry/attempt identity.

If the exact response body can be stored safely, preserve it as an immutable artifact.

Then exact artifact replay can use the stored response without spending another model call.

That is also useful for local-model workflows where model calls are expensive and repeated analysis of the same file should not require recomputation.

14. Retrieval Must Be Snapshot-Aware

Suppose the original run retrieved:

A.md
B.md
C.md

Three months later the index contains different chunks and embeddings.

Running the same query against the new index is not a replay.

Record:

query hash
retriever version
embedding model version
index snapshot id
candidate set
ranking scores
selected chunks
source content hashes

Then you can ask two distinct questions:

Original-evidence replay

What would the new agent do with exactly the evidence the old agent saw?

New-retrieval replay

What evidence would the new retriever obtain for the old task?

That decomposition is valuable because it separates retrieval regression from downstream reasoning regression.

15. Memory Needs Provenance Too

Long-lived agents often read memories that were written by previous runs.

A replay must know:

  • memory item ID,
  • write time,
  • source run,
  • provenance,
  • schema version,
  • content hash,
  • confidence/verification state,
  • promotion policy version,
  • tombstone/supersession state.

Otherwise replay can silently inject knowledge that did not exist when the original decision was made.

That is temporal leakage.

16. Verify the Exact Candidate

One of the most dangerous replay bugs is evidence detachment.

Example:

candidate A tested -> PASS
candidate B selected
run recorded -> PASS

This is invalid.

Verification evidence should bind to an exact candidate identity:

candidate_hash -> verifier_run -> evidence_hash -> outcome

If the candidate changes, the verification relationship must be invalidated or rerun.

17. Immutable Evidence Beats Editable Logs

Logs are operationally useful.

But an audit trail should not depend on mutable text files that can be rewritten without detection.

Useful options include:

  • append-only event streams,
  • immutable object storage,
  • content-addressed artifacts,
  • hash chains,
  • signed manifests,
  • database rows with immutable payload hashes.

The goal is not blockchain theater.

The goal is simple:

If the evidence changed, we should be able to tell.

18. Tamper-Evident Run Manifests

A simple manifest hash can bind all referenced artifacts:

@dataclass(frozen=True)
class ArtifactRef:
    kind: str
    artifact_id: str
    sha256: str


@dataclass(frozen=True)
class RunEvidence:
    run_id: str
    release_id: str
    artifacts: tuple[ArtifactRef, ...]
    previous_manifest_hash: str | None

You can then canonicalize and hash the manifest.

For stronger audit requirements, sign the manifest with an appropriate key-management process.

Again, the useful property is not sophistication.

It is detectability of mutation.

19. Replay Should Be Read-Only by Default

Never let a forensic replay accidentally resend emails, recreate cloud resources, merge code, modify customer records, or execute payments.

Default replay capabilities should be:

READ_RECORDED_ARTIFACTS
REEXECUTE_PURE_FUNCTIONS
RUN_SANDBOXED_TOOLS
COMPARE_POLICIES
RUN_VERIFIERS

not:

WRITE_PRODUCTION
SEND_EXTERNAL_MESSAGE
DELETE_RESOURCE
MERGE_CHANGE
DEPLOY

If side-effect replay is required, route it to an isolated test environment with explicit authorization.

20. A Replay Gateway

A useful implementation pattern is to separate tools from their recorded observations.

class ReplayGateway:
    def __init__(self, observations: dict[str, object]):
        self.observations = observations

    def call(self, operation_key: str):
        if operation_key not in self.observations:
            raise MissingReplayArtifact(operation_key)
        return self.observations[operation_key]

During forensic replay, the agent receives the historical observation.

During live execution, the normal gateway calls the real dependency.

During counterfactual replay, the candidate release can consume the historical evidence without touching production.

21. Missing Artifacts Must Be Explicit

A dangerous replay system silently fills gaps with live data.

Suppose the original browser snapshot is missing.

Do not quietly fetch the page today and continue.

Return something like:

REPLAY_INCOMPLETE
missing_artifact = browser_snapshot:obs-19

Then the operator can choose a weaker replay mode intentionally.

The system should know the difference between:

FULL_REPLAY
PARTIAL_REPLAY
LIVE_REVALIDATION
COUNTERFACTUAL_REPLAY

22. Counterfactual Replay for Policy Changes

Suppose release A escalated 42% of coding tasks to a frontier model.

Release B changes the routing threshold.

You can take old trajectories and ask:

Under the same observed evidence, where would B route differently?

That is useful.

But there is an important limit.

If B chooses an action A never executed, you may not know the outcome of that action.

For example:

old policy -> local model -> FAIL
new policy -> frontier model -> ?

Do not fabricate the counterfactual outcome.

Mark it unknown unless you can safely execute the missing branch offline.

23. Replay Search Decisions

For search-heavy agents, preserve:

  • node IDs,
  • parent IDs,
  • state hashes,
  • expansion policy version,
  • scorer version,
  • score vectors,
  • prune reasons,
  • cancellation reasons,
  • verifier outcomes,
  • branch cost.

Then you can inspect failures such as:

FAIL
└── selection failure
    └── winning branch existed
        └── pruned at depth 3
            └── scorer margin 0.02

That is much more actionable than:

agent failed

24. Replay Concurrency Carefully

Step 19 introduced speculative parallel execution.

Concurrency creates another reproducibility problem:

branch A finishes first today
branch B finishes first tomorrow

If selection depends on completion order, the system is schedule-sensitive.

Record:

  • launch time,
  • partial-result time,
  • completion time,
  • cancellation time,
  • selection deadline,
  • concurrency policy version.

Then provide two replay modes:

Historical schedule replay

Reproduce the original event ordering from recorded events.

Schedule-insensitive replay

Feed all completed results into deterministic selection order and see whether the outcome changes.

If it does, you have discovered schedule sensitivity.

25. Distributed Attempts Need Lineage

Step 20 separated logical operations from physical attempts.

Replay should preserve that structure:

operation-77
├── attempt-1 worker-A lease-9 timeout
├── attempt-2 worker-B lease-10 PASS
└── stale attempt-1 response rejected by fencing

Without this, duplicate work can look like contradictory history.

26. Replay Release Promotion Evidence

Step 24 introduced behavioral release manifests and promotion gates.

When a release is promoted, store the evidence used for promotion:

release candidate
   ↓
contract tests
   ↓
offline benchmark
   ↓
shadow comparison
   ↓
canary evidence
   ↓
promotion decision

Then six months later you can answer:

Why was this release considered safe to deploy?

That is release provenance, not just runtime provenance.

27. Provenance for Coding Agents

For a coding agent, an audit-ready run might include:

repository = org/service
base_commit = 81ac...
worktree_snapshot = sha256:...
request = task-44
behavior_release = agent-r17
planner_policy = plan-8
search_policy = mcts-5
tool_registry = tools-11
patch_candidate = sha256:...
pytest_output = artifact:194
lint_output = artifact:195
verifier = code-verifier-9
verdict = PASS
commit_operation = op-881

Now a regression can be reconstructed against the same source tree and verifier evidence.

28. Provenance for Research Agents

Record:

  • query,
  • source URLs,
  • retrieval timestamps,
  • content snapshots/hashes,
  • citation mapping,
  • source authority metadata,
  • model release,
  • claim-to-source edges,
  • verifier evidence.

This lets you distinguish:

source changed
retrieval changed
reasoning changed
citation mapping changed
verifier changed

Those are different failures.

29. Provenance for Browser Agents

Record:

  • account scope,
  • page snapshot,
  • action target identity,
  • element/DOM evidence,
  • idempotency key,
  • precondition,
  • action request,
  • response,
  • postcondition,
  • screenshots where useful.

For irreversible actions, provenance should make it obvious whether the action actually happened.

30. Provenance for Data Agents

Record:

input dataset snapshot
query/transform code hash
schema version
execution engine version
result hash
validation evidence
publication operation

Then a later discrepancy can be traced to data, code, engine, or validation changes.

31. Provenance for DevOps Agents

For infrastructure changes, preserve:

  • environment identity,
  • desired change,
  • observed pre-state,
  • generated plan,
  • approval identity,
  • tool/provider version,
  • fenced operation ID,
  • mutation request,
  • post-state,
  • health verification,
  • rollback target.

A statement like “the agent deployed it” is not enough provenance for a production incident.

32. Artifact Retention Has a Cost

Full replay data can become expensive.

You may store:

  • large prompts,
  • screenshots,
  • browser pages,
  • retrieved documents,
  • model outputs,
  • verifier artifacts,
  • test logs,
  • sandbox snapshots.

Use retention tiers.

For example:

Tier 0: hashes + metadata only
Tier 1: decision-critical artifacts
Tier 2: full verifier evidence
Tier 3: full forensic snapshot

Choose tier by risk, contract, incident status, regulation, and debugging value.

Do not retain sensitive artifacts merely because replay is useful.

33. Privacy and Secrets

Replay systems create a concentration of sensitive data.

Treat them accordingly.

Important controls include:

  • tenant isolation,
  • secret filtering,
  • field-level redaction,
  • access control,
  • retention limits,
  • encryption,
  • audit logs,
  • purpose limitation.

Do not store raw credentials for reproducibility.

Store stable references or safe fingerprints where possible.

34. Replay Quality Metrics

Track whether your replay system actually works.

Useful metrics include:

manifest completeness
artifact availability rate
hash verification rate
full replay success rate
partial replay rate
missing-artifact rate
schedule-sensitivity rate
counterfactual coverage
verifier-replay agreement
replay cost
replay latency

Also track which artifact classes are most often missing.

That tells you where your provenance design is weak.

35. Failure Taxonomy

A replay-aware agent platform should classify failures such as:

MISSING_RELEASE_ARTIFACT
MISSING_OBSERVATION
HASH_MISMATCH
MISSING_ENVIRONMENT_SNAPSHOT
MODEL_REVISION_UNAVAILABLE
TOOL_VERSION_UNAVAILABLE
RETRIEVAL_SNAPSHOT_UNAVAILABLE
MEMORY_TEMPORAL_LEAKAGE
VERIFICATION_EVIDENCE_DETACHED
SIDE_EFFECT_PROVENANCE_MISSING
SCHEDULE_SENSITIVITY
COUNTERFACTUAL_OUTCOME_UNKNOWN
REPLAY_POLICY_MISMATCH

This is much better than a generic replay failed.

36. Replay Invariants

Useful hard invariants include:

selected candidate must have exact candidate hash
PASS must reference verifier evidence
verifier evidence must reference exact candidate
side effect must reference operation ID
operation attempt must reference lease/fencing epoch when required
observation must reference source and snapshot
release ID must resolve to immutable manifest
replay must not silently call live dependencies
missing artifacts must remain explicit

These are ordinary software invariants.

They do not need an LLM.

37. A Minimal Replay Runner

A compact implementation might look like:

class ReplayRunner:
    def __init__(self, manifest_store, artifact_store, release_store):
        self.manifest_store = manifest_store
        self.artifact_store = artifact_store
        self.release_store = release_store

    def replay(self, run_id: str):
        manifest = self.manifest_store.get(run_id)
        release = self.release_store.get(manifest.release_id)

        artifacts = {
            ref.artifact_id: self.artifact_store.get(ref.artifact_id)
            for ref in manifest.artifacts
        }

        self._verify_hashes(manifest, artifacts)

        gateway = ReplayGateway(artifacts)

        return release.execute(
            task=artifacts[manifest.task_artifact_id],
            gateway=gateway,
            replay_mode=True,
        )

Production code needs far more nuance.

But notice the direction:

The replay runner uses immutable recorded evidence by default.

It does not casually call live production dependencies.

38. Do Not Use Replay to Manufacture Evidence

A counterfactual replay can suggest that a different policy might have behaved better.

That is not automatically production evidence.

If the replay substitutes recorded outputs, changes execution order, or omits unavailable side effects, mark those limitations.

The same evidence-first rule still applies:

A replay is an experiment whose assumptions must be explicit.

39. The Complete Evidence Loop

At this point the architecture forms a closed loop:

behavioral release
      ↓
production execution
      ↓
trajectory + provenance
      ↓
external verification
      ↓
immutable replay manifest
      ↓
offline replay / counterfactual analysis
      ↓
benchmark
      ↓
candidate release
      ↓
shadow / canary / promotion

That is much stronger than “we looked at the logs and changed the prompt”.

40. Do You Actually Need Full Replay?

Not every agent needs forensic reconstruction.

A small personal script may only need:

input
model version
prompt hash
tool outputs
final result

A production coding or infrastructure agent may need far more.

Use the same principle as the rest of this series:

Add the cheapest provenance mechanism that closes the failure you actually need to diagnose.

Do not build a distributed event-sourced audit platform for a five-line local automation.

But if an agent can change production systems, spend real money, modify repositories, or make externally visible decisions, being able to reconstruct its behavior is not optional observability.

It is part of correctness.

41. Practical Checklist

Before calling an agent run reproducible, ask:

  1. Can I identify the exact behavioral release?
  2. Can I recover the task bytes or their immutable reference?
  3. Can I recover the environment snapshot?
  4. Can I recover the exact external observations?
  5. Can I distinguish observation from inference?
  6. Can I reconstruct routing/search/budget decisions?
  7. Can I identify every consequential side effect?
  8. Can I bind verification to the exact candidate?
  9. Can I detect artifact mutation?
  10. Can I replay without touching production?
  11. Can I state which parts are nondeterministic?
  12. Can I distinguish replay from live revalidation?
  13. Can I identify missing artifacts explicitly?
  14. Can I compare an old trajectory against a new release?
  15. Can I preserve privacy while doing all of the above?

If several answers are no, the platform probably has logs, not provenance.

42. Final Principle

The most important idea is simple:

Reproducibility is not identical output. It is the ability to reconstruct the evidence, environment, policy, release, actions and verification that produced an outcome.

Advanced agents are probabilistic systems embedded in changing environments.

Perfect token determinism is often unavailable.

Operational reproducibility is still achievable.

Record immutable inputs.

Version the behavioral release.

Bind decisions to evidence.

Bind verification to exact candidates.

Bind side effects to operation identity.

Preserve lineage.

Make missing evidence explicit.

And never let a replay silently become a new live experiment.

The next stage is incident forensics for agent systems: turning provenance and replay into a disciplined process for reconstructing failures, identifying the earliest incorrect decision, measuring blast radius, distinguishing model error from system error, and producing remediation that can be verified against the original incident trajectory.