AI Prompts on Programmer.ie: Modern AI programming

Agent Architecture Review

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing an AI agent repository.

Do not judge the system by how many agent patterns it contains.
Reconstruct the actual execution loop and identify what each mechanism is doing.

Start by tracing one representative task end to end:

user/request
→ state construction
→ model call
→ candidate/action generation
→ validation
→ tool execution
→ observation
→ state update
→ stopping condition
→ final result
→ verification, if any

Identify the exact code responsible for each transition.

Then review these areas:

1. State
   - What state is authoritative?
   - What state exists only in model context?
   - Can the agent resume after process/model failure?
   - Are observations distinguished from assumptions?

2. Tool interfaces
   - Are tool schemas narrow and explicit?
   - Are malformed or unsafe calls rejected deterministically?
   - Does the model receive broader authority than the task requires?

3. Planning
   - Is planning genuinely separate from execution?
   - Does a plan constrain later action, or is it just prose?
   - Can the plan be revised when evidence changes?

4. Candidate generation and critique
   - Are multiple candidates or critique loops fixing a measured failure?
   - How are candidates selected?
   - Is the selector independent enough to add information?

5. Memory
   - What is stored?
   - Why is it retrieved?
   - Can stale or irrelevant memory alter current decisions?
   - Is memory treated as evidence rather than authority?

6. Search
   - Does the agent search over meaningful alternatives?
   - What objective evaluates branches?
   - What stops search?
   - Is search more effective than another model call under equal cost?

7. Feedback loop
   - Are tool results converted into structured observations?
   - Can the agent detect that an action failed?
   - Are retry and replanning separate concepts?

8. Verification
   - Who decides whether the task succeeded?
   - Is success checked outside the same model that generated the answer?
   - Are PASS, FAIL and UNKNOWN distinct?

9. Stopping
   - What causes the loop to stop?
   - Can it terminate because the model merely says it is done?
   - Are resource/budget limits explicit?

10. Complexity
   For every major agent mechanism, identify:
   - the failure it is intended to solve
   - evidence that the failure occurs
   - evidence that the mechanism improves it
   - operational cost introduced

For every finding, report:
- location
- observed behavior
- failure mode
- severity
- evidence strength
- smallest corrective mechanism
- verification method

End with:

A. Actual agent execution graph
B. Strong correctness boundaries
C. Places where the model is being trusted as operational truth
D. Agent mechanisms that have demonstrated value
E. Agent mechanisms whose value is currently unproven
F. Verification gaps
G. The minimum architecture you would test against the current design

Do not recommend more agent machinery by default.
Agent complexity is not a capability ladder.

Audit a Learned Scorer Before You Trust Its Score

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing a learned scoring model in this repository.

Do not begin by proposing a larger architecture. First reconstruct the scoring contract the model currently implements.

Inspect the repository and identify:
1. every input that reaches the scorer;
2. how those inputs are encoded or embedded;
3. where representations are combined;
4. the scalar target being predicted;
5. how training labels or preferences are produced;
6. the loss function and weighting;
7. how the score is consumed downstream;
8. the baseline or non-learned alternative, if one exists.

Build a scorer contract with:
- semantic meaning of each input;
- tensor/representation boundary;
- target definition;
- valid score range;
- calibration assumptions;
- downstream decision threshold or ranking behavior.

Then test the following failure modes:
- target leakage;
- proxy labels that do not match the downstream decision;
- duplicate or redundant features;
- scale mismatch between embeddings;
- hidden dependence on length, frequency or other shortcuts;
- train/inference feature mismatch;
- score used as probability without calibration evidence;
- one scalar being asked to represent several incompatible notions of quality.

For each finding provide exact file/symbol evidence and classify it as confirmed defect, plausible risk, or unsupported hypothesis.

Do not recommend adding layers, attention, recurrence or another model unless the current scorer fails a measurable requirement that the added mechanism is intended to solve.

Output:
1. Scorer contract
2. Training-target reconstruction
3. Downstream score usage
4. Evidence-backed findings
5. Shortcut/leakage risks
6. Simplest viable baseline
7. Smallest experiments needed
8. Recommendation: Keep / Simplify / Change

A score is useful only if its learned target matches the decision the system later makes.

Audit Agent Authority and Human Escalation Boundaries

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing authority boundaries in a production AI-agent system.

Do not answer with "add a human in the loop".
Reconstruct exactly what authority the system delegates and where that authority is enforced.

Inventory every action that can create an externally meaningful effect, including where applicable:
- repository writes,
- pull-request creation/merge,
- deployments,
- database mutations,
- refunds/payments,
- email/messages,
- account/permission changes,
- credential operations,
- infrastructure restarts,
- destructive actions,
- irreversible or regulated operations.

For each action build an authority table:
action | proposer | executor | required permission | approval rule | evidence shown to reviewer | enforcement location | bypass path | rollback/recovery

Then classify each action by:
- reversibility,
- financial/operational impact,
- blast radius,
- sensitivity,
- confidence/verification requirement,
- whether escalation is mandatory.

Inspect whether authority is enforced outside the model. Look specifically for:
- prompt-only safety rules,
- tools exposed even when policy should forbid them,
- model-controlled approval flags,
- broad credentials shared across tasks,
- permission checks after execution rather than before,
- approval gates that accept only model summaries,
- reviewers who cannot see diffs/evidence/side effects,
- stale approvals reused after the plan or state changes,
- no binding between approved intent and executed action,
- escalation that can be silently skipped after retries/replanning.

For each human approval point, determine whether the reviewer receives enough authoritative evidence to make an independent decision rather than rubber-stamping model narration.

Recommend the minimum authority architecture needed:
- explicit capability/permission boundary,
- pre-execution policy gate,
- scoped credentials,
- intent/action binding,
- evidence package for escalation,
- approval expiry/versioning,
- post-action verification.

Separate competence from authority. A model may be capable of an action without being permitted to take it.

Finish with a matrix of:
autonomous | escalate | prohibited
and explain the policy reason for each classification.

Audit Agent Replay and Provenance Completeness

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing whether this production agent system supports meaningful deterministic replay and provenance.

Do not treat ordinary logs as sufficient evidence of reproducibility.
Reconstruct the exact evidence required to explain a historical run after the surrounding system has changed.

Inspect the repository and identify how a run records or binds:
- run/trajectory identity,
- behavioral release/version,
- model/provider/model version,
- prompt/template version,
- policy/router version,
- tool schemas and implementation versions,
- retrieved documents and their immutable identities,
- memory inputs,
- external observations,
- environment/configuration snapshot,
- search/branch lineage,
- scheduler/worker attempts,
- side effects,
- verifier version and evidence,
- final outcome.

Build a provenance matrix with columns:
artifact or decision | current identifier | immutable? | persisted? | replayable? | missing dependency

Then select one representative agent run path and determine whether a future investigator could answer:
1. What did the agent actually observe?
2. Which code/prompt/model/policy produced each decision?
3. Which attempts executed versus merely being proposed?
4. Which external state was read?
5. Which side effects occurred?
6. Why was the result accepted or rejected?
7. Which parts can be replayed exactly and which require simulation/substitution?

Look for hidden drift such as:
- mutable prompt names without content hashes,
- model aliases instead of exact versions,
- retrieval IDs that resolve to current rather than historical content,
- memory references without snapshots,
- logs that omit rejected branches or retries,
- verifier PASS/FAIL without verifier evidence/version,
- side effects recorded only in model narration,
- run records that reference mutable configuration,
- timestamps without stable event ordering,
- missing idempotency/attempt identity.

Distinguish:
- deterministic replay,
- evidence replay,
- behavioral reconstruction,
- best-effort reproduction.
Do not claim a stronger property than the repository supports.

Recommend the minimum provenance contract needed to make failures scientifically inspectable. For each missing field, explain which future question becomes impossible without it.

The goal is not perfect token-level determinism. The goal is enough immutable evidence to reconstruct what the system saw, decided, executed and verified.

Audit Dependency Containment in a Production Agent

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing a production AI-agent system for dependency-failure containment.

Do not begin by recommending more retries, larger timeouts, more workers, or a different provider.
First reconstruct how dependency failure propagates through the system.

Inspect the repository and identify every material external or shared dependency used during agent execution, including where applicable:
- model providers,
- browser pools,
- retrieval/search services,
- embedding services,
- databases,
- queues,
- sandbox workers,
- verifier services,
- third-party APIs,
- shared caches,
- internal RPC services.

For each dependency, build a table containing:
1. caller / call site,
2. timeout,
3. retry policy,
4. retry ownership,
5. concurrency limit,
6. queue or worker pool used,
7. failure classification,
8. circuit-breaker behavior,
9. fallback or degradation behavior,
10. whether failure can consume capacity needed by healthy work.

Then trace at least one realistic failure path end to end:

dependency slows or fails
→ timeout
→ retry
→ queue/worker occupancy
→ upstream retry or replanning
→ additional load
→ system-wide effect

Look specifically for:
- retries at multiple layers multiplying attempts,
- no retry budget,
- identical policy for transient and permanent failures,
- unbounded queues,
- shared pools that allow one dependency to starve unrelated work,
- missing circuit breakers,
- circuit breakers without half-open recovery logic,
- fallback paths that silently weaken correctness,
- degraded modes that are not visible to verification,
- timeout values longer than the useful lifetime of the operation,
- model-driven retries that bypass system retry policy.

Classify each finding as:
- confirmed from code/config,
- strongly implied,
- requires runtime evidence.

Propose the minimum containment architecture needed. Prefer explicit retry budgets, bulkheads, bounded queues, dependency health state, circuit breakers and declared degradation over generic resilience advice.

For every recommended change, specify:
- failure mode it addresses,
- metric or experiment that would validate it,
- what behavior should occur when the dependency remains unhealthy.

Do not call the system resilient merely because retries exist. A resilient system must prevent a local dependency failure from consuming the control and execution capacity required by healthy work.

Audit Fast and Slow Learned State

Mon, 01 Jan 0001 00:00:00 +0000

Review this repository for hierarchical or recurrent learned state, especially architectures with fast/slow state, low/high-level state, recurrent refinement, or repeated latent updates.

Do not assume recurrence or hierarchy creates reasoning. Reconstruct what information each state is expected to retain and why one-pass computation is insufficient.

For each recurrent state identify:
- initialization;
- update equation/module;
- update frequency;
- inputs available at each update;
- reset boundary;
- detach/truncation behavior;
- persistence across examples or sequences;
- downstream consumers.

Build a state-timescale table and test:
- fast and slow states carrying the same information;
- one state dominating while the other is ignored;
- stale state crossing sequence/example boundaries;
- hidden leakage from future information;
- unstable or vanishing recurrent updates;
- iteration count chosen without evidence;
- repeated computation that converges immediately or never converges;
- hierarchy whose benefit disappears under an equal-budget feed-forward baseline.

Use exact file/symbol evidence. Distinguish measured behavior from architectural intent.

Output:
1. Recurrent state map
2. Fast/slow semantic contract
3. Reset and persistence analysis
4. Failure risks with evidence
5. Simpler equal-budget baseline
6. Ablations: remove hierarchy / reduce iterations / share state / one-pass
7. Verification metrics
8. Recommendation: Keep / Simplify / Remove hierarchy

Hierarchy earns its place only if distinct timescales produce measurable behavior a simpler model cannot match under comparable budget.

Audit Goals, Commitments and Executable Work in a Long-Running Agent

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing a long-running agent or workflow system for semantic confusion between goals, plans, commitments, tasks and actions.

Do not assume a plan step means work has been scheduled or promised.
Reconstruct the actual state machine used by the repository.

Identify the representations for:
- user/system goal,
- proposed plan,
- selected plan,
- commitment or obligation,
- queued task,
- leased/running work,
- completed action,
- verified outcome,
- cancellation,
- failure/retry,
- supersession/replanning,
- handoff to another worker or human.

Build a state-transition table:
state | created by | durable? | side effect permitted? | cancelable? | retryable? | completion evidence | successor states

Trace one representative long-running operation from intent to verified completion.

Look for semantic collapses such as:
- plan text directly becoming executable work,
- queued task treated as completed commitment,
- retry creating duplicate commitments,
- replanning leaving old tasks live,
- cancellation not propagating to leased work,
- worker completion treated as business outcome without verification,
- model narration used as state transition,
- no distinction between logical task ID and execution attempt ID,
- handoff losing ownership/accountability,
- commitments that cannot be enumerated or discharged.

For each externally visible commitment, answer:
1. Who created it?
2. What exact intent/version does it bind to?
3. What must happen for it to be discharged?
4. What happens if the plan changes?
5. Can retries duplicate the obligation?
6. Can another process recover it after a crash?

Recommend the minimum durable model needed to separate:
goal → plan → commitment → task → attempt → action → verification

Prefer explicit persisted states and identifiers over inferred status from logs or model text.

Finish with failure scenarios for:
- crash after commitment but before enqueue,
- duplicate delivery,
- stale plan after state change,
- cancellation during execution,
- verifier failure after side effect,
- worker death and resume.

For each scenario, state whether the current design is correct, ambiguous, or unsafe, and cite the repository evidence.

Audit Learned Component Boundaries

Mon, 01 Jan 0001 00:00:00 +0000

Review the learned component boundaries in this repository.

The question is not whether the architecture can be decomposed into separate models. The question is whether those boundaries improve correctness, evidence, ownership or independent evolution enough to justify their cost.

Trace each learned submodel end to end and record:
- inputs and outputs;
- training data and target;
- optimizer/update path;
- checkpoint lifecycle;
- inference caller;
- dependencies on other learned components;
- whether it can be evaluated independently.

For every boundary ask:
1. Does this component solve a distinct prediction problem?
2. Does it require different data or optimization?
3. Does it need independent deployment/versioning?
4. Can its quality be measured independently?
5. Would a shared encoder/head produce the same observable behavior more simply?

Check for:
- duplicate encoders with no demonstrated benefit;
- separate models that are always trained and deployed together;
- implicit circular dependence between outputs;
- stale checkpoints or version skew;
- duplicated preprocessing;
- component APIs that hide incompatible semantic assumptions;
- separate models created only because the concept has a separate name.

Cite exact repository evidence for every conclusion.

Output:
1. Learned component graph
2. Boundary justification table
3. Shared versus independent training analysis
4. Version/deployment coupling risks
5. Boundaries that should remain
6. Boundaries that could collapse
7. Smallest ablations needed
8. Recommendation with expected simplification benefit

A component boundary should correspond to a real independence property, not a diagramming preference.

Audit Parallel Agent Work for Speculation, Isolation and Safe Cancellation

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing an agent system that performs work in parallel or speculatively.

Your job is to determine whether concurrency improves useful exploration without creating duplicate side effects, shared-state corruption, unbounded cost, or false assumptions about cancellation.

Do not recommend more parallelism merely because independent tasks exist.
First reconstruct what currently runs concurrently and what authority each branch has.

1. Map concurrent work.
   Identify:
   - fan-out points
   - worker/task creation
   - candidate generation
   - parallel tool calls
   - background jobs
   - early-stop/cancellation logic
   - join/selection points

2. Classify every parallel branch.
   For each branch report:
   - purpose
   - independence assumptions
   - mutable state it can access
   - external side effects it can create
   - compute/tool budget
   - cancellation mechanism
   - evidence required before its output is selected

3. Check speculative isolation.
   Look for:
   - branches sharing the same mutable workspace
   - writes to production/external systems before selection
   - shared caches or memory that allow losing branches to contaminate winners
   - generated artifacts without stable branch/run identity
   - concurrent operations that assume ordering accidentally

4. Check cancellation semantics.
   Determine whether "cancelled" means:
   - cancellation was requested
   - worker coroutine/task stopped
   - underlying API/tool call stopped
   - external side effect did not occur

   Flag code that treats these as equivalent without evidence.

5. Check commit/selection boundaries.
   Consequential effects should ideally occur only after a candidate/branch has been selected and revalidated.
   Identify paths where speculative work can commit irreversible or duplicate effects early.

6. Check duplicate-work control.
   Distinguish intentional independent exploration from accidental duplication caused by retries, queues, restarts or multiple coordinators.

7. Check budget behavior.
   Determine whether parallelism can multiply:
   - model tokens
   - tool calls
   - browser sessions
   - workspaces
   - verification cost
   beyond the intended run budget.

8. Check verification and stale-state risk.
   Before a winning speculative branch commits, does the system revalidate:
   - current intent
   - current state
   - authority
   - branch ownership
   - external preconditions

For every issue report:

- file and symbol
- evidence
- failure mode
- blast radius
- confidence
- smallest corrective mechanism
- test/failure injection that would prove the correction

Return:

A. Concurrency map
B. Safe speculative paths
C. Shared-state / side-effect hazards
D. Cancellation assumptions that are not actually guaranteed
E. Duplicate-work risks
F. Budget amplification risks
G. Minimum architecture changes, ordered by value

Governing rule:
Parallelize independent, low-risk, high-value work; isolate speculative state; cancel aggressively when evidence is strong; commit side effects only after selection and revalidation.

Audit PyTorch Attention Shapes, QKV and Masks

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing a PyTorch attention or transformer implementation for dimensional and masking defects.

Do not assume a runtime-successful attention block is correct. Attention can execute while attending to the wrong positions, mixing heads incorrectly or applying masks with unintended broadcasting.

Trace one complete attention call from input embeddings through output projection.

Work in this order:

1. Record the dimensional contract for:
   - input embeddings
   - Q, K and V projections
   - head split
   - attention scores
   - masks
   - softmax weights
   - weighted values
   - head concatenation
   - output projection

Use semantic names such as batch, heads, query_length, key_length and head_dim rather than raw integers only.

2. Verify invariants:
   - embed_dim == num_heads * head_dim where required
   - Q/K inner dimensions align for score computation
   - attention scores have the intended [batch, heads, query, key] semantics
   - V multiplication returns the intended per-head representation
   - concatenation reconstructs the embedding dimension correctly

3. Inspect every reshape/view/transpose/permute and identify whether its semantic layout matches the next operation.

4. Audit masks separately:
   - causal mask orientation
   - padding mask meaning
   - boolean versus additive masks
   - mask rank and broadcasting
   - query length versus key length
   - cross-attention differences
   - masked values before softmax
   - all-masked rows producing NaNs

5. Compare custom attention code with any higher-level PyTorch API usage and flag incompatible assumptions about batch_first, mask shapes or return layouts.

Classify findings as:
- confirmed dimensional defect
- confirmed masking defect
- silent semantic risk
- fragile assumption
- correct behavior

For each issue provide:
- file/function
- expected shape/meaning
- actual transformation
- why it is wrong
- smallest safe fix
- a targeted test using a tiny deterministic attention example

Output:

## Attention shape ledger
A step-by-step table from input through output.

## Mask semantics
State exactly which positions are allowed and forbidden.

## Findings
Ranked by severity.

## Minimal corrections
No architecture redesign unless required by a proven invariant violation.

## Verification
Include shape assertions plus at least one tiny mask test where expected attention eligibility is known in advance.

Audit PyTorch Autograd and Gradient Flow

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing a PyTorch repository for autograd and gradient-flow defects.

Do not start by changing the optimizer, learning rate, architecture or loss function. First establish whether the intended parameters are actually connected to the loss through a valid computation graph.

Trace one complete training step from model input to optimizer.step().

Work in this order:

1. Identify:
   - the loss tensor
   - trainable parameters
   - optimizer parameter groups
   - zero_grad / backward / step ordering
   - gradient accumulation behavior

2. Trace the graph-breaking risks:
   - detach()
   - torch.no_grad()
   - .item() used too early
   - tensor reconstruction via torch.tensor(existing_tensor)
   - conversion through NumPy/Python values
   - requires_grad changes
   - frozen parameters that should be trainable
   - parameters omitted from the optimizer
   - in-place operations that invalidate gradient history

3. Inspect backward semantics:
   - repeated backward through freed graphs
   - retain_graph used to mask design problems
   - gradient accumulation without intentional scaling
   - zeroing gradients at the wrong time
   - multiple losses with unintended graph interaction

4. Inspect numerical gradient failures:
   - NaN / Inf loss
   - NaN / Inf gradients
   - exploding norms
   - mixed precision misuse
   - unsafe divisions/logs/exponentials
   - masking that creates invalid values

5. Classify every finding as:
   - confirmed graph break
   - confirmed optimizer/gradient lifecycle defect
   - confirmed numerical instability
   - suspicious but unproven
   - valid behavior

For each real issue provide:
- file/function
- the exact graph or lifecycle break
- affected parameters
- evidence from the code
- smallest corrective change
- a test or runtime probe proving gradients now flow correctly

Output:

## Training-step graph
Describe the actual loss → backward → parameter update path.

## Findings
Ranked by severity and confidence.

## Minimal corrections
Do not broaden beyond demonstrated failures.

## Verification
Include checks such as parameter grad presence, finite-gradient assertions, gradient norms and before/after parameter deltas.

If autograd is correctly wired, say so and identify which gradient assumptions deserve tests.

Audit PyTorch CUDA Memory and Training Performance

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing a PyTorch repository for CUDA memory problems or poor training performance.

Do not start by reducing batch size, enabling torch.compile, adding mixed precision, or changing the model. First determine which resource is actually limiting the workload.

Inspect the training/inference loop, model construction, optimizer, data transfer, precision policy, logging/metrics and any profiling code.

Work in this order:

1. Establish workload context:
   - device(s)
   - model/parameter size where inferable
   - optimizer
   - batch and sequence/image sizes
   - dtype / autocast policy
   - gradient accumulation
   - checkpointing
   - torch.compile usage

2. Reconstruct GPU memory ownership:
   - parameters
   - gradients
   - optimizer state
   - activations
   - temporary tensors
   - cached allocator memory
   - tensors accidentally retained across iterations

3. Inspect CUDA OOM risks:
   - references preserving computation graphs
   - loss/output tensors appended without detach
   - unnecessary activation retention
   - oversized temporary tensors
   - sequence/batch growth
   - fragmentation versus true capacity pressure
   - optimizer-state cost
   - validation/inference accidentally building graphs

4. Inspect performance risks:
   - CPU/data starvation
   - frequent cuda synchronize behavior
   - .item() or CPU transfers in the hot loop
   - excessive tiny kernels
   - repeated device allocations/copies
   - non_blocking used without the conditions required for overlap
   - logging/profiling that forces synchronization
   - poor batch sizing
   - compilation graph breaks or recompilation
   - mixed precision that is absent, misapplied or numerically unsafe

5. Separate the diagnosis into:
   - memory capacity bound
   - allocator/retention defect
   - data/input bound
   - CPU/synchronization bound
   - GPU compute bound
   - kernel/launch overhead bound
   - compile/runtime graph issue
   - insufficient evidence

6. For every proposed optimization, state:
   - evidence supporting the bottleneck
   - expected mechanism of improvement
   - likely trade-off
   - measurement that would confirm success

Output:

## Resource map
Summarize workload, precision and major memory consumers.

## Findings
Rank confirmed defects separately from optimization opportunities.

## Measurement plan
Specify appropriate PyTorch profiler/timing/memory observations needed to discriminate competing hypotheses.

## Minimal interventions
Recommend the smallest experiment first. Examples may include fixing retained graphs, changing transfer behavior, precision, batch size, checkpointing or compilation — but only when justified.

## Verification
Compare before/after peak allocated memory, peak reserved memory, step time, throughput and GPU utilization where available.

Never claim torch.compile, AMP, larger batches or more workers will improve performance without measuring the result in this workload.

Audit PyTorch DataLoader and Input Pipeline Performance

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing a PyTorch repository where training throughput or GPU utilization may be limited by the input pipeline.

Do not assume DataLoader settings are the problem, and do not recommend increasing num_workers by default. First determine where each training iteration spends time.

Inspect the dataset, transforms, sampler, DataLoader construction, device transfer and training loop.

Work in this order:

1. Reconstruct the batch path:
   storage/source → Dataset.__getitem__ → transforms/collate → worker queue → host batch → device transfer → model.

2. Identify current settings and constraints:
   - batch_size
   - num_workers
   - pin_memory
   - persistent_workers
   - prefetch_factor
   - shuffle/sampler
   - custom collate_fn
   - CPU-heavy transforms
   - storage/network access
   - host-to-device transfer pattern

3. Look for evidence of stalls:
   - synchronous disk/network work per sample
   - repeated parsing/decoding that could be cached or moved
   - serialized Python transforms
   - expensive collate logic
   - too few or too many workers
   - worker startup repeated every epoch
   - unpinned host memory when asynchronous GPU transfer is intended
   - device copies performed in a way that forces synchronization
   - tiny batches that create excessive per-batch overhead
   - data preprocessing occurring on the critical training thread

4. Distinguish bottlenecks:
   - storage/I/O bound
   - CPU preprocessing bound
   - DataLoader scheduling/worker bound
   - host-to-device transfer bound
   - model/GPU bound
   - insufficient evidence

5. For every finding provide:
   - exact file/function
   - evidence
   - expected mechanism causing the delay
   - smallest experiment to prove or falsify it
   - only then, the smallest optimization

Output:

## Pipeline map
Show the actual batch path and relevant settings.

## Bottleneck hypotheses
Ranked by evidence, not intuition.

## Measurement plan
Specify timings/profiling that isolate data time, transfer time and compute time.

## Recommended changes
Only changes whose mechanism is supported by evidence.

## Verification
Define before/after measures such as batches/sec, examples/sec, GPU utilization, data-wait time and epoch duration.

Do not call an optimization successful because one metric improved if total training throughput did not.

Audit PyTorch Tensor Shapes and Broadcasting Bugs

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing a PyTorch repository for tensor-shape and broadcasting defects.

Do not begin by rewriting the model or adding defensive reshapes. First reconstruct the dimensional contract the code is trying to implement.

Inspect the relevant data pipeline, model forward path, loss computation and any preprocessing/postprocessing that changes tensor dimensions.

Work in this order:

1. Identify the important tensors crossing subsystem boundaries.
   For each one, record:
   - semantic meaning of every dimension
   - expected shape
   - actual shape where it can be inferred
   - dtype and device when relevant

2. Trace every shape-changing operation, including:
   - reshape
   - view
   - flatten
   - squeeze / unsqueeze
   - transpose / permute
   - stack / cat
   - indexing and slicing
   - pooling
   - broadcasting arithmetic

3. Look specifically for high-risk failures:
   - batch and sequence dimensions swapped
   - accidental removal of batch dimension with squeeze
   - view() after a non-contiguous permutation
   - reshape that preserves element count but changes semantic layout
   - silent broadcasting that makes incompatible tensors appear valid
   - concatenation or stacking along the wrong axis
   - hard-coded dimensions that fail for different batch/sequence/image sizes
   - loss targets whose shape is accepted but semantically wrong

4. Separate findings into:
   - confirmed defect
   - likely defect
   - fragile dimensional assumption
   - valid but confusing code

5. For every confirmed or likely defect, provide:
   - exact file/function
   - input shape
   - transformation
   - resulting shape
   - why it is wrong
   - smallest safe correction
   - a runtime assertion or test that would prove the correction

Do not recommend broad refactors unless the evidence shows the current dimensional contract cannot be made reliable locally.

Output:

## Shape map
A compact table of the important tensors and their dimensional meaning.

## Findings
Ranked by severity, with repository evidence.

## Minimal corrections
Only changes justified by the findings.

## Verification
Concrete assertions/tests using representative and edge-case shapes.

If the shape flow is correct, say so explicitly and identify the assumptions that should remain documented or tested.

Audit Q, V, Policy and Advantage Objectives

Mon, 01 Jan 0001 00:00:00 +0000

Review this repository as a learned-decision system with multiple heads such as Q, V, policy, advantage, score, confidence, reward or related outputs.

Do not assume that more heads mean better modeling. Reconstruct why each head exists and what distinct failure it is supposed to solve.

For every head identify:
- exact input representation;
- prediction target;
- target provenance;
- loss function and coefficient;
- update frequency;
- gradient path into shared layers;
- downstream consumer;
- inference-time role.

Then build an objective matrix showing where heads share data, representations, labels and gradients.

Check for:
- two heads learning the same quantity under different names;
- advantage derived inconsistently from Q and V;
- policy targets that disagree with the value objective;
- one loss dominating shared representation learning;
- detached or accidentally coupled gradients;
- target leakage between heads;
- labels unavailable at inference being used indirectly as features;
- outputs computed but not used downstream;
- shared encoders forcing incompatible tasks into one representation.

For each issue cite exact files/symbols and separate confirmed behavior from inference.

Require evidence before recommending a separate network, extra head, auxiliary loss or more capacity.

Output:
1. Multi-head architecture map
2. Objective/target matrix
3. Gradient-sharing map
4. Redundant or conflicting heads
5. Leakage and proxy risks
6. Heads that earn their place
7. Smallest ablations or experiments
8. Recommendation: Keep / Merge / Separate / Remove

Each learned output must correspond to a distinct decision need, not merely an architectural possibility.

Audit Recursive Compute in a Small Model

Mon, 01 Jan 0001 00:00:00 +0000

Review this repository for recursive or iterative use of a compact learned model.

Do not treat repeated inference as reasoning by default. Determine what changes from one iteration to the next and whether additional iterations improve the target behavior under an equal compute budget.

Trace the recursive loop and identify:
- initial state/input;
- model applied at each step;
- state passed forward;
- stopping rule;
- maximum iteration budget;
- supervision applied per step or only at the end;
- gradient path through iterations;
- inference-time iteration policy.

Measure or derive, where repository evidence allows:
- output/state delta per iteration;
- prediction quality by iteration;
- compute cost by iteration;
- convergence or oscillation behavior;
- sensitivity to initialization;
- whether later iterations merely repeat the first result.

Check for:
- fixed iteration counts with no empirical justification;
- recurrence that adds latency but no measurable gain;
- unstable state growth;
- accidental detach preventing intended learning;
- exploding memory from full unrolled graphs;
- stopping decisions made by the same unverified signal being optimized;
- a larger one-pass model outperforming recursion at equal FLOPs/latency;
- a smaller iteration budget matching full performance.

Cite exact file/symbol evidence and separate observations from hypotheses.

Output:
1. Recursive execution trace
2. Per-iteration state/output contract
3. Compute-versus-quality curve or instrumentation plan
4. Convergence/stability risks
5. Equal-budget baselines
6. Smallest ablations
7. Recommended stopping policy
8. Recommendation: Keep / Reduce / Replace recursion

Recursive computation earns its place only when later steps add measurable information or correction rather than merely repeating work.

Audit the Control Plane of a Production AI Agent

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing the control-plane architecture of a production AI-agent system.

The core question is:
Who controls the system that controls the model?

Do not accept prompt instructions or model self-restraint as an enforceable control plane.
Inspect the repository and identify where the following decisions are made and enforced:
- intent admission,
- competence/model selection,
- tool/action authority,
- budgets and quotas,
- placement/worker selection,
- retry policy,
- dependency health/degradation,
- release/model/prompt/policy version selection,
- security/credential scope,
- human escalation,
- cancellation/kill switches,
- verification requirements,
- final completion authority.

Build a control matrix:
control concern | decision owner | enforcement component | model can influence? | model can bypass? | persisted/auditable? | failure behavior

Then reconstruct the execution path for one representative action:
intent
→ admission/control decision
→ reasoning/planning
→ authority check
→ scheduling/execution
→ observation
→ verification
→ completion

Mark every point where probabilistic model output is converted into an authoritative system transition.

Look specifically for architecture where the model effectively acts as:
- permission system,
- scheduler,
- retry controller,
- budget manager,
- release manager,
- health monitor,
- source of truth for execution success,
- verifier of its own work,
- escalation authority.

Also look for control logic duplicated inconsistently across agents/workers/tools.

For each control concern, determine whether it is:
- explicit and enforceable,
- explicit but advisory,
- implicit,
- delegated to the model,
- missing.

Recommend the minimum explicit control-plane boundary needed. Preserve probabilistic reasoning where it is useful, but move policy and authority into inspectable system components.

The target architecture should make this distinction concrete:
Execution proposes.
Control permits.
External systems execute/observe.
Verification establishes outcome.

Do not recommend a large orchestration framework merely because it exists. Add control mechanisms only where a specific failure or authority problem requires them.

Finish with:
1. current control-plane diagram,
2. bypass paths,
3. highest-risk model-controlled responsibilities,
4. minimum remediation sequence,
5. tests that prove the model cannot exceed policy even when it proposes an invalid action.

Audit Whether This Agent Has a Safe and Unambiguous Tool Interface

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing the tool boundary of an AI agent.

Your job is to determine whether the model is being given a clear, typed and safe action space or is being forced to guess between overlapping tools and ambiguous schemas.

Do not begin by changing prompts or model choice.
First reconstruct the tool surface from code.

Work in this order:

1. Inventory every model-callable tool
For each tool record:
- name,
- description,
- parameter schema,
- required vs optional fields,
- return schema,
- side effects,
- authority level,
- error modes,
- retry/idempotency characteristics.

2. Audit semantic overlap
- Identify tools whose descriptions or capabilities overlap.
- Identify vague verbs such as search, lookup, fetch, run, execute, update, process.
- Check whether the model can distinguish discovery vs exact lookup, read vs write, preview vs commit, local vs remote, query vs mutation.
- Find parameters whose meaning depends on undocumented conventions.

3. Audit schemas
- Check whether enums could replace free text.
- Check whether mutually exclusive arguments are represented explicitly.
- Identify oversized schemas where most fields are irrelevant to most calls.
- Identify nullable/default behavior that can cause unintended actions.
- Check validation before execution.

4. Audit routing
- Find where a proposed action is mapped to a tool.
- Determine whether routing is model-only, deterministic, constrained, or post-validated.
- Check whether tool availability is scoped by task, state, competence, or authority.
- Identify fallback behavior after invalid calls.

5. Audit results
- Determine whether tool results are typed enough for the agent to distinguish success, partial success, no result and failure.
- Check whether prose result text can be misread as authoritative state.
- Identify missing IDs, version markers, timestamps, status codes, changed-state evidence or provenance.

6. Audit side-effect safety
- Separate read tools from mutation tools.
- Identify destructive or externally visible tools.
- Check confirmation/authority gates, idempotency keys and postconditions.
- Determine whether retries can duplicate writes.

7. Adversarial cases
Reason through:
- two tools both plausibly match the same request,
- required semantic distinction is only in a long description,
- model omits an optional field whose default is dangerous,
- tool returns partial success,
- malformed result still looks like success,
- retry repeats an external mutation.

Output:

A. Tool inventory
B. Ambiguity/overlap matrix
C. Schema weaknesses
D. Routing weaknesses
E. Result-contract weaknesses
F. Side-effect and retry risks
G. Smallest interface corrections
H. Concrete routing/schema tests

Classify each finding as:
- CONFIRMED FROM CODE
- STRONGLY INFERRED
- NEEDS RUNTIME EVIDENCE

Treat tool design as action-space design.
A stronger model does not fix an ambiguous action space.

Audit Whether This Agent Knows When to Stop

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing an AI agent that can iterate, call tools, revise work, or continue until a task is complete.

Your job is to determine whether the loop has explicit progress semantics and defensible stopping conditions.

Do not assume retries are progress.
Do not assume model confidence is completion.

Trace one representative agent run through:
observe -> decide -> act -> observe -> repeat/stop.

Work in this order:

1. Identify loop state
- Find the durable or in-memory state carried between iterations.
- Record what changes after every action.
- Identify counters, budgets, timestamps, completed steps, failed attempts, tool results, and external state references.

2. Identify progress signals
- Find how the system determines whether an iteration improved the task state.
- Separate activity from progress.
- Check whether repeated equivalent actions are detected.
- Check whether the system can recognize no-op edits, identical tool results, unchanged browser state, or repeated failed hypotheses.

3. Identify stopping conditions
Enumerate every stop path:
- verified success,
- hard budget exhaustion,
- retry limit,
- no-progress threshold,
- unrecoverable failure,
- human escalation,
- cancellation,
- timeout,
- model-issued stop.

For each, determine who decides and what evidence is required.

4. Audit budgets
- Token/model-call budget
- Tool-call budget
- wall-clock/runtime budget
- financial cost budget if present
- retry budget per failure class
- search/branch budget if relevant

Check whether nested loops can bypass the outer budget.

5. Audit termination reasons
- Determine whether the final state records WHY the run ended.
- Check whether success, failure, timeout, cancellation and budget exhaustion are distinguishable downstream.
- Check whether callers can accidentally interpret an incomplete run as success.

6. Failure injection
Reason through:
- same tool called with equivalent arguments repeatedly,
- repeated edits toggle code back and forth,
- external state never changes,
- model keeps proposing new wording without new evidence,
- one subtask loops while global budget appears healthy,
- success condition is never externally verified,
- cancellation arrives between action and observation.

Output:

A. Loop state model
B. Progress-signal table
C. Stop-condition table
D. Budget escape paths
E. Infinite-loop / false-progress risks
F. Smallest corrections
G. Tests for no-progress, timeout, cancellation and success termination

Classify findings as:
- CONFIRMED FROM CODE
- STRONGLY INFERRED
- NEEDS RUNTIME EVIDENCE

The key question is not whether the loop has a maximum iteration count.
It is whether the agent can distinguish useful state change from repeated activity.

Audit Whether This Agent Memory Is Relevant, Fresh and Trustworthy

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing the memory system of an AI agent.

Your job is to determine whether stored information actually improves future decisions and whether retrieval can safely distinguish prior evidence from current truth.

Do not equate persistence with useful memory.
Do not equate retrieved text with authoritative state.

Work in this order:

1. Inventory memory classes
Identify where the repository stores:
- current runtime/task state,
- working memory,
- semantic facts,
- episodic run history,
- procedural rules/workflows,
- exact or approximate caches,
- embeddings/vector indexes,
- summaries or compressed history.

For each, record ownership, lifetime, write path and read path.

2. Trace one memory write
For one representative stored item record:
- source event/data,
- original timestamp,
- author/system/model origin,
- confidence/provenance if any,
- scope (user/project/repository/task/global),
- version or state identity,
- transformation/summarization before storage.

3. Trace one retrieval
Follow a future decision that consumes prior memory.
Record:
- retrieval query,
- candidate generation,
- ranking/filtering,
- freshness checks,
- scope checks,
- conflict handling,
- how retrieved memory enters the model or control logic.

4. Audit relevance
- Determine whether retrieved memories demonstrably affect a decision.
- Check for high-volume history that adds context but no decision value.
- Identify retrieval that favors lexical similarity over task relevance.
- Check whether failed prior attempts are distinguishable from successful knowledge.

5. Audit freshness and truth
- Identify memories that can become stale when repository files, remote services, user intent or external state change.
- Check whether authoritative state is re-read when needed.
- Identify cached outputs whose validity depends on model version, prompt, file hash, tool version or environment.
- Check conflict resolution between old memory and current observation.

6. Audit contamination
- Can one user/project/task retrieve another scope's memory?
- Can speculative model output become durable fact without verification?
- Can summaries erase uncertainty or provenance?
- Can a bad prior run poison later decisions?

7. Failure injection
Reason through:
- file changed after memory was stored,
- prior run failed but its summary sounds authoritative,
- same semantic fact has conflicting versions,
- cache key omits an input that changed,
- retrieval returns relevant but obsolete information,
- memory grows until useful evidence is buried.

Output:

A. Memory architecture map
B. Write-path provenance table
C. Retrieval decision trace
D. Freshness/staleness risks
E. Scope and contamination risks
F. Cache-binding gaps
G. Smallest corrections
H. Tests that demonstrate memory improves decisions safely

Classify each finding as:
- CONFIRMED FROM CODE
- STRONGLY INFERRED
- NEEDS RUNTIME EVIDENCE

The standard is not 'the agent remembers more.'
The standard is: prior information changes a future decision for the better without replacing authoritative current state.

Audit Whether This Agent Search Strategy Is Worth Its Extra Compute

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing an AI agent that explores more than one candidate action, plan or trajectory.

Your job is to determine whether search is implemented as a disciplined decision process and whether it earns its latency and compute cost.

Do not assume more branches mean better reasoning.
Do not assume a model-generated score is an objective measure of branch quality.

Work in this order:

1. Reconstruct the search topology
- Identify where branching occurs: candidate answers, plans, actions, tool paths or full trajectories.
- Record branching factor, depth, beam width, retry count and maximum live states.
- Identify whether states are copied, shared or mutated across branches.

2. Identify the state representation
For every search node determine what is preserved:
- task state,
- observations,
- tool results,
- files/external side effects,
- model context,
- accumulated cost,
- score,
- parent/lineage,
- verification status.

3. Audit candidate generation
- Determine whether branches are genuinely diverse or near-duplicates.
- Check temperature/prompt/tool/state differences that create diversity.
- Identify deterministic bottlenecks that make multiple branches equivalent.

4. Audit scoring and pruning
- Find every scorer, critic, heuristic or verifier used to rank branches.
- Determine what evidence each score consumes.
- Check for self-scoring where the same model generates and judges without external signal.
- Identify premature pruning that can eliminate slower but better trajectories.
- Check tie-breaking and score normalization.

5. Audit side effects and isolation
- Determine whether speculative branches can mutate shared files, databases, browser state or remote systems.
- Check whether branches use sandboxes/worktrees/snapshots or another isolation mechanism.
- Identify side effects that cannot be rolled back when a branch loses.

6. Audit budgets
Measure or reconstruct:
- model calls,
- tokens,
- wall-clock time,
- tool calls,
- external requests,
- memory/state growth.

Check whether the budget is global and whether nested search can bypass it.

7. Compare against simpler baselines
Require an equal-budget comparison against at least:
- single-path execution,
- Best-of-N at the final answer only,
- shallower/narrower search where appropriate.

Compare task success, verified correctness, latency and cost.

8. Failure injection
Reason through:
- all branches repeat the same early mistake,
- scorer prefers polished but incorrect trajectories,
- losing branch already produced an irreversible side effect,
- branch state leaks into another branch,
- best branch is pruned early,
- search budget explodes on one difficult task.

Output:

A. Search topology
B. Search-state contract
C. Generation-diversity assessment
D. Scoring/pruning audit
E. Isolation and side-effect risks
F. Budget model
G. Equal-budget baseline plan
H. Smallest corrections
I. Tests/benchmarks required to justify keeping search

Classify every finding as:
- CONFIRMED FROM CODE
- STRONGLY INFERRED
- NEEDS RUNTIME EVIDENCE

The standard is not 'search is more sophisticated.'
The standard is: under a comparable budget, search produces measurably better verified outcomes often enough to justify its complexity.

Audit Whether This Agent Separates Planning From Execution

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing a repository that implements a multi-step AI agent.

Your job is to determine whether the system has a real planner/executor boundary or merely lets one model improvise while calling the result a plan.

Do not recommend a new planner architecture until you reconstruct the current control flow from code.

Inspect the repository and trace one representative task from user goal to final completion.

Work in this order:

1. Reconstruct the planning boundary
- Find where goals are converted into steps, tasks, dependencies, or a plan object.
- Identify whether the plan is structured data or only natural-language text.
- Record which component owns planning and which component owns execution.
- Identify whether execution can mutate, skip, reorder, or invent plan steps.

2. Build a plan contract
For each planned step record:
- step identifier,
- intended action,
- required inputs,
- prerequisite state,
- expected output,
- completion condition,
- failure condition,
- whether the executor can verify it independently.

3. Trace state between planner and executor
- Determine what repository/runtime state the planner sees.
- Determine what state the executor sees when the step actually runs.
- Find stale-plan risks: changed files, changed remote state, changed tool outputs, changed user intent.
- Identify whether preconditions are checked again before execution.

4. Audit replanning
- Find every path that can trigger replanning.
- Determine whether replanning is based on new evidence or merely model dissatisfaction.
- Check whether already-completed work is preserved or repeated.
- Check whether failed steps can create endless plan-regenerate-plan loops.

5. Audit authority
- Determine whether planning grants execution authority implicitly.
- Identify destructive or high-impact actions that can be planned and then executed without a separate gate.
- Separate: plan proposes -> control permits -> executor acts.

6. Failure injection
Reason through at least these cases:
- a prerequisite step failed but later steps remain runnable,
- repository state changes after planning,
- tool output contradicts a planning assumption,
- executor partially completes a step,
- plan contains an impossible or underspecified action,
- replanning produces a conflicting replacement plan.

Output:

A. Current control-flow trace
B. Planner/executor boundary map
C. Step contract table
D. Stale-plan and replanning failure modes
E. Authority leaks
F. Smallest architectural corrections
G. Concrete tests that prove the corrections

For every finding label it as one of:
- CONFIRMED FROM CODE
- STRONGLY INFERRED
- NEEDS RUNTIME EVIDENCE

Do not treat a model-generated list of steps as evidence that the system has a planner.
A plan is useful only if execution can interpret, constrain, update and verify it as system state.

Audit Whether This Agent Verifies Success Outside the LLM

Mon, 01 Jan 0001 00:00:00 +0000

You are auditing an AI agent repository for false-success risk.

The central question is:
Where can the agent say "done" without an independent system establishing that the intended postcondition is actually true?

Do not treat model confidence, self-critique, a successful tool call, or a plausible final answer as verification.

Inspect the repository in this order:

1. Identify completion paths.
   Find every place a task/run can enter a terminal success state.
   Record:
   - file and symbol
   - success status/value emitted
   - evidence required before that transition

2. Reconstruct the task contract.
   For each major task type identify:
   - intended external outcome
   - authoritative system of record
   - observable postcondition that would prove success

3. Classify current evidence.
   For every success path classify the strongest evidence available as one of:
   - model assertion
   - model critique/judge
   - tool response
   - deterministic local check
   - authoritative external read
   - independent verifier

4. Find false-PASS paths.
   Look for cases where:
   - the model marks its own work complete
   - a tool returning HTTP/command success is treated as business success
   - generated code is accepted without tests/static checks/runtime evidence
   - browser actions are assumed to have taken effect without rereading state
   - writes are considered successful without checking the authoritative resource
   - UNKNOWN/ambiguous outcomes are collapsed into PASS

5. Check verifier independence.
   Determine whether the verifier:
   - observes the result independently
   - has access to authoritative state
   - can return PASS / FAIL / UNKNOWN
   - is insulated from persuasive generator rationale where practical

6. Check failure handling.
   What happens when verification is unavailable, contradictory, stale or ambiguous?
   Flag any fail-open behavior that converts missing evidence into success.

7. Propose the minimum verification boundary.
   For each high-value false-success risk, specify:
   - exact postcondition
   - authoritative evidence source
   - deterministic check where possible
   - when an LLM-based semantic verifier is genuinely needed

Return:

A. Completion-state map
B. Confirmed false-PASS risks
C. Weak or non-independent verification
D. Missing authoritative postconditions
E. Places where UNKNOWN must remain UNKNOWN
F. Smallest verification changes to implement first
G. Tests that prove the agent can no longer claim success incorrectly

Governing rule:
The agent may produce the action, but it does not get to define reality.

Choose the Simplest Model Architecture That Fits This Repository

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing a machine-learning repository to decide whether its current model architecture matches the decision problem it actually has.

Do not rank architectures by sophistication.
Do not assume that adding value heads, policies, hierarchy, recurrence, recursive refinement or multiple learned components is an improvement.

First reconstruct the actual decision being learned.

1. Identify the model's contract.
   - What inputs does it receive?
   - What output does downstream software consume?
   - Is it estimating a scalar score, comparing alternatives, predicting value, selecting an action, maintaining state, refining an internal representation, or combining several of these?

2. Identify every learned component.
   For each component report:
   - inputs
   - outputs
   - training target / supervision source
   - loss
   - where its output affects the final decision

3. Reconstruct the information flow.
   Draw a concise dependency graph from input → learned components → decision/output.
   Separate learned computation from deterministic software.

4. Identify the minimum sufficient model class.
   Ask in order:
   - Would one scalar scorer solve the real decision?
   - Does the system genuinely need separate value/Q/policy semantics?
   - Is temporal/recurrent state required by the data or only present architecturally?
   - Does hierarchy correspond to measurable structure in the problem?
   - Does recursive/refinement computation improve a measurable outcome?
   - Are multiple learned heads providing distinct information or duplicating each other?

5. Test every additional mechanism against evidence.
   For each architectural component beyond the minimum, identify:
   - the failure it is intended to fix
   - evidence that the failure exists
   - benchmark or ablation showing the component helps
   - training/inference/maintenance cost it adds

6. Look for removable complexity.
   Flag components that:
   - have no independent target or clear semantic role
   - are never meaningfully consumed downstream
   - duplicate deterministic logic
   - cannot be validated independently
   - add state without evidence state is required
   - survive only because the architecture was copied from another system

7. Look for missing capability only after simplification.
   If the current architecture cannot represent the real decision, describe the smallest additional learned mechanism that would address the demonstrated gap.

Return:

A. Actual decision problem
B. Current learned-component graph
C. Minimum sufficient architecture
D. Components that earn their complexity
E. Components that should be ablated or removed
F. Missing evidence before changing architecture
G. One benchmark/ablation plan that can decide the question

The governing rule is:
Use the simplest model that solves the decision you actually have.

Debug Why This PyTorch Model Is Not Learning

Mon, 01 Jan 0001 00:00:00 +0000

You are debugging a PyTorch model that runs but is not learning correctly.

Do not begin by changing the architecture, optimizer, learning rate, or model size.
First determine where learning actually breaks.

Inspect the repository and trace one complete training path from batch creation to parameter update.

Work in this order:

1. Establish the symptom.
   - What metric or loss indicates that learning is failing?
   - Is the failure flat loss, divergence, oscillation, overfitting, underfitting, NaNs, or apparently correct training with bad evaluation?
   - Identify the exact training command/configuration if available.

2. Trace the data.
   - Inspect dataset construction, transforms, batching, labels/targets, masks and splits.
   - Verify representative input and target shapes, dtypes, ranges and devices.
   - Look for train/eval leakage or preprocessing differences.

3. Trace the forward pass.
   - Record important tensor shapes through the model.
   - Check accidental broadcasting, wrong dimensions, silent reductions, masking errors and activation assumptions.

4. Trace the loss.
   - Verify output and target semantics match the chosen loss.
   - Check logits versus probabilities, label encoding, reduction, masking and class dimensions.
   - Identify whether the observed loss scale is plausible.

5. Trace autograd.
   - Identify trainable parameters expected to receive gradients.
   - Look for detach(), no_grad(), tensor recreation, in-place operations or frozen parameters that break the graph.
   - Determine whether gradients are absent, zero, exploding, NaN or merely small.

6. Trace the optimizer update.
   Verify the actual order of:
   - zero/reset gradients
   - forward
   - loss
   - backward
   - clipping, if any
   - optimizer step
   - scheduler step

   Confirm the optimizer owns the intended parameters.

7. Compare train and evaluation modes.
   - Check model.train() / model.eval().
   - Check dropout, batch normalization, inference-mode behavior and metric calculations.

8. Only after correctness checks, inspect optimization choices.
   Consider learning rate, scheduler, initialization, batch size, normalization and capacity only if the earlier path is coherent.

For every finding provide:

- file and symbol
- observed evidence
- why it can prevent or distort learning
- confidence: high / medium / low
- smallest test or change that can confirm the diagnosis

End with:

A. Most likely root cause
B. Confirmed defects
C. Plausible but unproven causes
D. Experiments to run next, ordered by information value
E. Architectural changes that should NOT be attempted yet

Do not declare the problem solved because the code looks conventional.
Do not recommend a larger model or more training until the existing learning path has been verified.

Model Architecture Review

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing a repository that contains one or more learned models.

Your job is not to praise the architecture or replace it with a fashionable one.
Your job is to reconstruct what the learned system actually does and identify where the architecture does or does not match the decision it is supposed to solve.

Start by building an architecture inventory.

For every learned component, identify:
- its input
- its output
- what target or signal trains it
- where its output is consumed
- whether it scores, predicts, ranks, routes, values, classifies, generates, or controls
- whether it is used at training time, inference time, or both

Then reconstruct the model graph:

raw input
→ representation / encoder
→ learned components
→ intermediate scores or latent state
→ decision rule
→ final output

Do not infer this from class names alone. Trace actual call paths.

Review the architecture for these questions:

1. Is the learned target aligned with the real decision?
   Example failure:
   the system trains a proxy score but production chooses using a different objective.

2. Are multiple learned models genuinely solving different subproblems?
   Or are several models duplicating the same signal?

3. Is a learned component being used where a deterministic rule would be clearer, cheaper, and more reliable?

4. Is a deterministic rule being used where the problem is genuinely semantic or uncertain and would benefit from learning?

5. Are intermediate scores calibrated enough for how they are used?
   Look for thresholds, ranking assumptions, comparisons across models, and unvalidated probability-like outputs.

6. Does the system confuse:
   - prediction with value
   - value with policy
   - ranking with classification
   - confidence with correctness
   - recurrence with reasoning
   - hierarchy with additional depth

7. Are model outputs composed in a principled way?
   Inspect:
   - weighted sums
   - gates
   - routers
   - learned selectors
   - recursive calls
   - recurrent state
   - hierarchical aggregation

8. Is complexity justified by evidence?
   For every additional model/head/router/recurrent layer, identify the measurable failure it is intended to fix.

9. Could the architecture be simplified without removing demonstrated capability?

10. Is there a benchmark that isolates whether each architectural component contributes value?

For every architectural finding, report:
- component(s)
- code location
- observed behavior
- intended role, if documented
- mismatch or risk
- evidence strength
- simplest experiment that would confirm or reject the concern

End with:

A. Architecture map
B. Learned components and their real jobs
C. Strongly justified components
D. Components whose value is currently unproven
E. Deterministic substitutions worth testing
F. Missing experiments
G. The simplest architecture that appears capable of solving the demonstrated problem

Do not assume a more sophisticated model is better.
Use the simplest model that solves the decision the system actually has.

Production Agent Architecture Review

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing a production AI agent or autonomous engineering system.

Treat the model as one component inside a larger operational system.
Do not assume that stronger reasoning compensates for weak control, verification, security, recovery, or distributed-systems correctness.

Reconstruct one consequential task from current intent to externally verified outcome.

Trace:

intent
→ goals / work decomposition
→ competence decision
→ authority decision
→ placement
→ workflow state
→ reasoning / search
→ candidate action
→ verification
→ mutation gate
→ external side effect
→ postcondition verification
→ provenance / reliability evidence

Then audit the following boundaries.

1. Intent
   - Is intent versioned?
   - Can cancelled or superseded work still mutate external systems?
   - Are consequential operations bound to the current intent version?

2. Authoritative state and freshness
   - Which assumptions can become stale?
   - Are exact state identities, versions, SHAs, ETags, epochs, or hashes checked before mutation?
   - Can approvals or verification survive changes that should invalidate them?

3. Competence
   - What evidence says the system is competent for this task regime?
   - Is competence measured or inferred from model confidence?
   - What happens outside the demonstrated competence envelope?

4. Authority
   - What is the maximum action the agent can perform?
   - Is authority scoped to the task, resource, tenant, operation and time window?
   - Can delegation or tool choice silently expand authority?

5. Trust boundaries
   - Can user text, web pages, retrieved documents, tool output, memory, generated code, or another agent become control instructions?
   - Are credentials kept outside model context where possible?
   - Are generated artifacts quarantined before privileged execution?

6. Verification
   - Is outcome verification independent of generation?
   - Are PASS, FAIL and UNKNOWN distinct?
   - Can a permissive or degraded verifier produce false success?

7. Mutation gateway
   Before an external side effect, does the system re-check:
   - current intent
   - state freshness
   - authority
   - competence
   - security scope
   - ownership/fencing
   - release/policy version
   - budget
   - verification evidence
   - idempotency

8. Durable workflows
   - Can work survive worker/model/process restarts?
   - Are waits, retries, timers, approvals and external events durable?
   - Are model calls activities rather than the source of workflow truth?

9. Distributed execution
   - Are leases, fencing and idempotency explicit?
   - Can retries duplicate consequential effects?
   - Can stale workers continue mutating after ownership changes?

10. Transactions and recovery
   Classify side effects as:
   - reversible
   - compensatable
   - irreversible
   - unknown

   For partial failure, identify whether the system can:
   - verify reality
   - compensate
   - roll forward
   - reconcile ambiguous outcomes

11. Multi-agent coordination
   - Is ownership explicit?
   - Is commitment ownership different from task delegation?
   - Are messages evidence/proposals rather than authority?
   - Is consensus being mistaken for verification?

12. Reliability
   - What are the production SLIs/SLOs?
   - Are false PASS, FAIL and UNKNOWN measured separately?
   - Do error budgets affect release or autonomy decisions?

13. Behavioral releases
   - Can you identify the exact model, prompts, tools, policies, routers, memory/retrieval configuration, workflow definition and verifiers used for a run?
   - Can the run be reconstructed later?

14. Complexity
   For every advanced mechanism, ask:
   - what measured failure does it fix?
   - what is the verified improvement?
   - what new failure modes does it introduce?
   - could deterministic software solve this more cheaply?
   - could the mechanism be removed today without losing demonstrated capability?

For every finding, report:
- subsystem and code location
- concrete evidence
- violated or missing invariant
- consequence / blast radius
- evidence strength
- smallest corrective action
- how the correction should be externally verified

End with:

A. End-to-end production execution map
B. Strongest existing control boundaries
C. Highest-risk trust/authority/verification gaps
D. Recovery and distributed-correctness gaps
E. Reliability and provenance gaps
F. Complexity that is demonstrably justified
G. Complexity that has not yet earned its place
H. Minimum production architecture that should be benchmarked against the current system

Do not reward architectural sophistication for its own sake.
The goal is externally verified useful behavior under the authority, reliability, and security constraints that actually matter.

PyTorch Repository Review

Mon, 01 Jan 0001 00:00:00 +0000

You are reviewing a PyTorch repository as an evidence-driven ML engineer.

Do not begin by proposing a rewrite.
First inspect the repository and build a concrete picture of how training actually works.

Review the code in this order:

1. Identify the model entry points.
   - model classes
   - forward paths
   - training entry points
   - evaluation/inference entry points

2. Trace tensor shapes through the important paths.
   Look for:
   - accidental broadcasting
   - incorrect reshape/view/permute usage
   - batch/sequence/channel dimension confusion
   - shape assumptions that are not enforced

3. Inspect autograd behavior.
   Look for:
   - unintended detach() calls
   - in-place operations that can damage gradient computation
   - parameters that should receive gradients but do not
   - tensors tracking gradients unnecessarily
   - retained graphs or references that can grow memory use
   - incorrect gradient accumulation or zeroing

4. Inspect the training loop.
   Verify the actual order of:
   - forward pass
   - loss calculation
   - backward pass
   - optimizer step
   - gradient reset
   - scheduler step, if present

5. Inspect the loss and targets.
   Check whether:
   - output shapes match target expectations
   - activation/loss combinations are appropriate
   - reductions are intentional
   - masking is correct
   - training and evaluation compute comparable quantities

6. Inspect data loading and preprocessing.
   Look for:
   - unnecessary Python-side work
   - repeated transformations
   - incorrect batching
   - device-transfer bottlenecks
   - train/eval leakage
   - nondeterministic behavior that matters to debugging

7. Inspect device and precision handling.
   Look for:
   - repeated CPU↔accelerator transfers
   - tensors/models on different devices
   - unnecessary synchronization
   - unsafe or ineffective mixed precision
   - places where memory use is larger than necessary

8. Inspect performance only after correctness.
   Identify evidence for:
   - low accelerator utilization
   - avoidable small operations
   - excessive Python loops
   - repeated allocations
   - recomputation
   - opportunities for batching or compilation

For every finding, report:

- location: file and symbol
- evidence: what the code actually does
- consequence: correctness, learning, memory, or performance
- confidence: high / medium / low
- smallest reasonable fix
- how to verify the fix

Separate the final report into:

A. Confirmed correctness problems
B. Likely training-quality problems
C. Performance opportunities
D. Things that look unusual but are probably intentional
E. Missing evidence / questions that must be answered before changing code

Do not recommend an optimization merely because PyTorch supports it.
Do not recommend architectural changes unless the current implementation demonstrates a measurable failure that the change would address.

Reconstruct Why This Agent Run Failed

Mon, 01 Jan 0001 00:00:00 +0000

You are performing incident forensics on a failed or incorrect AI-agent run.

Do not begin with the final error message or the final bad output.
Do not accept the agent's own explanation as root cause.
Reconstruct the run from persisted evidence and find the earliest divergence that materially changed the outcome.

Use available repository code plus run artifacts/log schemas to reconstruct:
- initial intent and inputs,
- behavioral release identity,
- observations,
- planner/router decisions,
- candidate/search branches,
- tool calls and results,
- retries and worker attempts,
- state transitions,
- memory reads/writes,
- side effects,
- verifier decisions and evidence,
- final reported result.

Produce a chronological incident table:
sequence | component | input | decision/action | observed result | expected invariant | divergence? | evidence

Then identify separately:
1. first anomalous observation,
2. first incorrect decision,
3. first irreversible or externally visible side effect,
4. first point where recovery became impossible or unlikely,
5. first verifier/control failure that allowed the bad trajectory to continue.

Do not collapse these into one event unless the evidence supports that.

Classify candidate causes as:
- root cause,
- contributing condition,
- amplification mechanism,
- detection failure,
- downstream symptom.

Look for common false diagnoses:
- blaming the model when the observation was stale,
- blaming a tool when routing supplied invalid inputs,
- blaming a retry when the operation lacked idempotency,
- blaming verification when the verifier never received authoritative state,
- blaming the final step when the plan was already invalid several steps earlier,
- treating correlation in timestamps as causation.

For the leading root-cause hypothesis, provide:
- exact supporting evidence,
- contradicting evidence,
- confidence,
- what additional evidence would falsify it.

Then test the proposed remediation counterfactually:
Would this fix have prevented the same run from failing at the earliest divergence?
If not, do not call it a root-cause fix.

Finish with:
- root-cause statement,
- blast radius,
- missing observability/provenance that made diagnosis harder,
- minimal remediation,
- regression/replay test that would prove the remediation works.

Separate confirmed evidence from inference throughout.