Use with AI · Path Review

Model Architecture Review

Map a learned system into its actual model components, decision boundaries and training signals before judging the architecture.

Path Review Models From First Principles Model and reasoning repositories Intermediate
View the learning path →

How to use this

  1. Open a repository-aware AI assistant.
  2. Give it access to the repository or files you want reviewed.
  3. Copy the prompt below and run it unchanged first.
  4. Use the evidence it finds to decide what to inspect or change next.
PromptCopy and run against your own project
You are reviewing a repository that contains one or more learned models.

Your job is not to praise the architecture or replace it with a fashionable one.
Your job is to reconstruct what the learned system actually does and identify where the architecture does or does not match the decision it is supposed to solve.

Start by building an architecture inventory.

For every learned component, identify:
- its input
- its output
- what target or signal trains it
- where its output is consumed
- whether it scores, predicts, ranks, routes, values, classifies, generates, or controls
- whether it is used at training time, inference time, or both

Then reconstruct the model graph:

raw input
→ representation / encoder
→ learned components
→ intermediate scores or latent state
→ decision rule
→ final output

Do not infer this from class names alone. Trace actual call paths.

Review the architecture for these questions:

1. Is the learned target aligned with the real decision?
   Example failure:
   the system trains a proxy score but production chooses using a different objective.

2. Are multiple learned models genuinely solving different subproblems?
   Or are several models duplicating the same signal?

3. Is a learned component being used where a deterministic rule would be clearer, cheaper, and more reliable?

4. Is a deterministic rule being used where the problem is genuinely semantic or uncertain and would benefit from learning?

5. Are intermediate scores calibrated enough for how they are used?
   Look for thresholds, ranking assumptions, comparisons across models, and unvalidated probability-like outputs.

6. Does the system confuse:
   - prediction with value
   - value with policy
   - ranking with classification
   - confidence with correctness
   - recurrence with reasoning
   - hierarchy with additional depth

7. Are model outputs composed in a principled way?
   Inspect:
   - weighted sums
   - gates
   - routers
   - learned selectors
   - recursive calls
   - recurrent state
   - hierarchical aggregation

8. Is complexity justified by evidence?
   For every additional model/head/router/recurrent layer, identify the measurable failure it is intended to fix.

9. Could the architecture be simplified without removing demonstrated capability?

10. Is there a benchmark that isolates whether each architectural component contributes value?

For every architectural finding, report:
- component(s)
- code location
- observed behavior
- intended role, if documented
- mismatch or risk
- evidence strength
- simplest experiment that would confirm or reject the concern

End with:

A. Architecture map
B. Learned components and their real jobs
C. Strongly justified components
D. Components whose value is currently unproven
E. Deterministic substitutions worth testing
F. Missing experiments
G. The simplest architecture that appears capable of solving the demonstrated problem

Do not assume a more sophisticated model is better.
Use the simplest model that solves the decision the system actually has.