The Safe but Useless Model

This is not a universal AI score. It is useful only where the domain has an explicit exclusion structure.

A simpler benchmark may just report:

A rate
B rate
HYBRID rate
ABSTAIN rate

under a task contract that declares which states are admissible.

The key distinction remains:

LEGITIMATE HYBRID
both actions are jointly feasible and justified

HYBRID EVASION
the task requires commitment under scarcity,
but the model refuses to choose

13. Decision extraction is another sensor

All of the previous measurements assume that we can map free-form output into a structured decision.

That mapping:

A decision extractor therefore needs its own contract:

decision_extractor_contract = {
    "input": "free_text_recommendation",
    "output_classes": [
        "PRESERVE_CASH",
        "INVEST_FOR_GROWTH",
        "HYBRID",
        "ABSTAIN",
        "UNKNOWN",
    ],
    "method": "rubric_or_structured_extractor",
    "validation": {
        "human_audit": True,
        "report_accuracy": True,
        "report_agreement": True,
    },
    "known_blind_spots": [
        "implicit_decision",
        "conditional_recommendation",
        "multi-stage_plan",
        "rhetorical_paraphrase",
    ],
}

If $g(Y)$ is wrong, the basin analysis is wrong.

This is another instance of the book’s recurring rule:

Every observable needs a measurement contract.

For production systems, it is often useful to make the generator expose a structured decision payload directly:

from pydantic import BaseModel
from typing import Any

class DecisiveFact(BaseModel):
    variable_name: str
    observed_value: Any
    expected_direction: str | None = None

class ReversalBoundary(BaseModel):
    target_variable: str
    threshold_condition: str
    expected_new_decision: str

class StructuredDecisionPayload(BaseModel):
    primary_decision: str
    decisive_facts: list[DecisiveFact]
    tradeoff: str | None = None
    rejected_alternatives: dict[str, str]
    reversal_boundaries: list[ReversalBoundary]
    confidence: float | None = None
    is_hybrid_choice: bool = False

Structured output does not force correctness. It makes the decision and its claimed dependencies inspectable.

14. Basin occupancy is only a marginal diagnostic

For discrete decisions:

But $ΔB$ is only a marginal statistic.

Suppose the oracle says:

scenario 1 → A
scenario 2 → A
scenario 3 → B
scenario 4 → B

and the model says:

scenario 1 → B
scenario 2 → B
scenario 3 → A
scenario 4 → A

Both distributions have dominant occupancy $0.5$.

The model is still wrong on every case.

So Chapter 10 needs three separate diagnostics:

MARGINAL CONVERGENCE
How concentrated are model decisions?
→ basin occupancy / entropy

SCENARIO AGREEMENT
Did each scenario receive the required decision?
→ oracle agreement

CONTEXT COUPLING
Does changing the decisive context move the decision distribution?
→ paired intervention response

Never substitute the first for the other two.

15. The scenario × decision matrix is the primary basin artifact

For scenario families such as:

DISTRESS
STABLE
GROWTH

and decisions:

PRESERVE
OPTIMIZE
INVEST

report the conditional decision matrix:

A context-insensitive system may instead produce something like:

These numbers are illustrative, not results from our own experiment.

The real artifact should be compared with the oracle-required matrix and accompanied by uncertainty intervals.

For a deterministic extracted decision, scenario-level agreement is:

16. Mutual information measures context coupling, not correctness

For controlled scenario classes $X$ and extracted decisions $D$, one can estimate:

That makes mutual information a useful context-coupling diagnostic.

One caveat is easy to get wrong. The plug-in estimator of $I(X;D)$ from empirical counts is biased upward, and the bias grows with the number of cells (scenario classes times decision classes) relative to the sample size. With a handful of samples per scenario, a model that ignores context can still post a positive $\hat I(X;D)$ from noise alone. Report a bias-corrected estimate or a permutation baseline — shuffle the scenario labels, recompute, and check that the observed $\hat I$ sits well above that null distribution — before reading any coupling into the number.

One may also compare against the oracle:

But this is not a correctness metric.

A model can map each scenario class deterministically to the wrong decision and still have high mutual information.

So:

I(X;D)
→ how much decisions depend on context class

oracle agreement
→ whether that dependence is correct

Context utilization is relational. Dependence without directional correctness is not enough.

17. Match the distance to the decision object

Whole-answer cosine similarity is useful for exploration but often wrong for the actual decision.

Use a distance appropriate to the structured unit:

The same prose can encode different decisions, and different prose can encode the same decision.

Measure the smallest object that contains the expected effect.

18. Stochastic control uses decision distributions

For a stochastic generator:

Estimate instead:

The experiment must also report whether the mass moved in the oracle-required direction.

For each intervention family report:

samples per condition
within-condition decision entropy
paired decision-change rate
direction-correct rate
scenario-level oracle agreement
confidence intervals
perturbation validity

That separates true context sensitivity from sampling noise.

19. Context length and context utilization are different properties

A common response to generic output is:

Add more context.

Sometimes that works.

But a 5,000-token prompt is not evidence that the model used the five facts that determine the decision.

A direct test compares, for example:

SHORT
3 months runway
negative cash flow

LONG
5,000-token company description
containing the same decisive facts

Then intervene on the same decisive fact in both conditions.

The question is not:

Did the long prompt produce a longer answer?

It is:

Did the decisive variable have more, less, or the same directional effect on the decision?

The issue is not context length.

The issue is decision leverage.

20. Build a domain-specific anti-genericity benchmark

This chapter does not hand you one universal TrendslopScore.

It gives you the specification required to build a falsifiable benchmark for your domain.

A minimum suite might contain:

Decision-relevant inversions

runway: 18 months ↔ 3 months
budget: $10M ↔ $100k
market: growth ↔ contraction
deadline: 1 year ↔ 1 week
risk tolerance: high ↔ near-zero
regulation: permissive ↔ prohibitive

Context ablations

remove budget
remove runway
remove deadline
remove market direction
remove decisive evidence

Invariance controls

paraphrase
format
entity names
irrelevant ordering

Trade-off tests

A
B
HYBRID
ABSTAIN

with an explicit admissibility contract.

Claimed-rationale tests

For every self-reported decisive fact:

intervene
regenerate
test directional response

Reversal tests

For every declared reversal boundary:

instantiate boundary
regenerate
verify reversal

A compact benchmark design could use:

6 inversion types × 2 directions = 12 paired conditions
5 ablations
4 invariance controls
3 samples per condition

The exact count depends on cost and domain. What matters is that the transformation family and oracle are frozen before final evaluation.

Required reporting should include:

perturbation validity rate
decision-extractor accuracy/agreement
scenario-level oracle agreement
context acknowledgment rate
paired decision-change rate
direction-correct rate
within-condition entropy
basin occupancy
excess basin occupancy
hybrid/non-choice rate
decisive-fact faithfulness
reversal-condition fidelity

This is enough to turn:

the answer feels generic

into a reproducible measurement problem.

Our earlier internal exploration of the Trendslop hypothesis supplied an initial perturbation taxonomy, semantic-divergence sketches, and response-manifold intuition. It did not produce the controlled book-owned benchmark required by the stricter Chapters 9–10 protocol.

That is not a reason to invent a number.

It is a specification for the next experiment.

21. Production reality: use the surface selectively

Full counterfactual evaluation is expensive.

A base prompt plus five perturbations and three samples per condition already requires eighteen generations.

That may be appropriate for a benchmark. It may be absurd as a synchronous gate.

Separate:

CHEAP SYNCHRONOUS
containment
structured constraints
runtime verification

SHADOW
counterfactual suites on sampled traffic

TEMPLATE / WORKLOAD LEVEL
representative prompt-family testing

RISK TRIGGERED
high-impact decisions
low decision specificity
high uncertainty
novel scenario family

OFFLINE REGRESSION
known counterfactual families after changes

MODEL / CONFIG RELEASE GATE
full sensitivity suite before promoting
model, prompt, retrieval, or policy versions

A compact asynchronous shadow check can be as simple as:

import asyncio

async def shadow_counterfactual_check(
    generate,
    extract_decision,
    base_prompt,
    counterfactual_prompt,
    expected_relation,
):
    base_text, cf_text = await asyncio.gather(
        generate(base_prompt),
        generate(counterfactual_prompt),
    )

    base_decision = extract_decision(base_text)
    cf_decision = extract_decision(cf_text)

    return {
        "base_decision": base_decision,
        "counterfactual_decision": cf_decision,
        "relation_passed": expected_relation(
            base_decision,
            cf_decision,
        ),
    }

This is an orchestration sketch, not a production-complete evaluator. Real systems still need repeated sampling, perturbation validation, extractor validation, tracing, and uncertainty estimates.

22. Repair the system, not only the prose

Once context-insensitive convergence is detected, several interventions are possible.

Force explicit trade-offs

Require:

one primary recommendation
one rejected alternative
reason for rejection

Require structured decision output

A typed payload makes:

decision
decisive facts
tradeoff
rejected alternatives
reversal boundaries

observable.

This can act as a useful mechanical wedge against vague hybrid output, but it does not guarantee correct reasoning.

Generate alternatives before selecting

Use the model to widen the option set, then evaluate alternatives under explicit constraints.

Counterfactually verify the recommendation

Intervene on a claimed decisive fact and test whether the decision moves correctly.

Separate proposal from policy

Let the model propose possibilities. Let a later policy or optimization layer decide what is admissible.

Escalate ambiguous trade-offs

If the available evidence cannot distinguish the options, the right response may be review or abstention rather than a forced answer.

The last intervention leads directly to Chapter 11.

23. The safe-but-useless signature in the reliability record

The diagnostic record can now expose a failure that static hallucination checks miss:

containment                 = PASS
relation_fidelity           = PASS
polarity_fidelity           = PASS
provenance                  = VERIFIED
invariance_controls         = PASS
context_acknowledgment      = PASS
counterfactual_decision     = FAIL
directional_fidelity        = FAIL
context_ablation            = FAIL
decisive_fact_faithfulness  = FAIL
reversal_condition_fidelity = FAIL
decision_specificity        = LOW

Nothing here says:

hallucination detected

Yet the output should not be trusted as a context-specific recommendation.

The pattern is:

containment = PASS
constraint fidelity = PASS
context sensitivity = FAIL

This is the signature Chapter 9 prepared us to observe.

24. What you should now be able to answer

After this chapter, you should be able to explain:

1. Why different wording is not evidence of different decisions.
2. Why basin occupancy is useful but cannot replace scenario-level oracle agreement.
3. How context ablation differs from counterfactual inversion.
4. Why mentioning a decisive fact does not prove the fact influenced the recommendation.
5. How to test a model’s stated reversal condition.
6. When a hybrid answer is a legitimate composition and when it is choice avoidance.
7. Why higher temperature does not establish greater context sensitivity.
8. Why more context tokens do not prove more context utilization.

Exercises

Exercise 1 — Build a two-basin test

Construct ten distress scenarios and ten growth scenarios with clear oracle decisions. Generate multiple responses per scenario, extract one structured decision, and report:

scenario × decision matrix
oracle agreement
basin occupancy
hybrid rate
direction-correct counterfactual rate

Exercise 2 — Test a claimed decisive fact

Ask a model to provide a recommendation plus the three facts it considers decisive. Change one claimed fact while holding the others fixed. Does the recommendation move in the predicted direction?

Exercise 3 — Execute a reversal condition

Require the model to state what condition would reverse its recommendation. Construct that condition and rerun the model. Record reversal-condition fidelity.

Exercise 4 — Separate diversity from context use

Run the same scenario at several decoding temperatures. Then run two decision-distinguishing scenarios at one fixed temperature. Compare lexical diversity with decision-distribution movement. Which change actually tracks context?

25. The deeper lesson

Generic answers are attractive because they survive many evaluators. They are hard to falsify, sound reasonable, avoid controversial choices, and often contain familiar best practices.

But decision support is not a contest in producing statements that are difficult to disagree with.

It is a task of conditioning recommendations on constraints.

The relevant distinction is:

GOOD GENERAL ADVICE
can be broadly useful across many cases

GENERIC COLLAPSE
persists even when the task contract
says the decision should change

So the engineering rule is:

Do not ask only whether an answer is reasonable. Ask which facts made this answer different from the one the system would have produced anyway.

And then test those facts.

A fact is not demonstrated to be decision-relevant because the model mentions it.

A fact is demonstrated behaviorally when changing it changes the decision in the relation the task requires.

If we cannot observe that dependency, we may not have observed context-sensitive reasoning.

We may have observed a polished default.

Research roots

1. Angelo Romasanta, Llewellyn D. W. Thomas and Natalia Levina, “Researchers Asked LLMs for Strategic Advice. They Got ‘Trendslop’ in Return,” Harvard Business Review, March 16, 2026. Reports persistent strategic biases across seven core business tensions and warns that LLM recommendations can follow fashionable managerial patterns rather than context-specific strategic logic. https://hbr.org/2026/03/researchers-asked-llms-for-strategic-advice-they-got-trendslop-in-return

2. Natalia Levina, “Researchers Asked LLMs for Strategic Advice. They Got ‘Trendslop’ in Return,” NYU Stern Research Highlight, March 16, 2026. Summarizes the study as thousands of simulations in which leading LLMs repeatedly selected trendy strategic options across varied contexts. https://www.stern.nyu.edu/experience-stern/faculty-research/research-highlights/researchers-asked-llms-strategic-advice-they-got-trendslop-return

3. Harvard Business Review, product summary for Romasanta, Thomas and Levina, “Researchers Asked LLMs for Strategic Advice. They Got ‘Trendslop’ in Return,” 2026. Practical guidance includes using LLMs to expand options rather than make final strategic choices, counteracting biases, avoiding the hybrid trap, and not assuming that context alone removes the bias. https://store.hbr.org/product/researchers-asked-llms-for-strategic-advice-they-got-trendslop-in-return/H093GG

4. Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” 2023. Finds sycophancy across several state-of-the-art assistants, shows that human preference data can favor answers matching user views, and reports that preference-model optimization can sometimes trade truthfulness for sycophancy. https://arxiv.org/abs/2310.13548

5. Ethan Perez et al., “Discovering Language Model Behaviors with Model-Written Evaluations,” Findings of ACL 2023, pp. 13387–13434. Uses model-written evaluations to identify behaviors including sycophancy and reports examples where RLHF exacerbated undesirable behaviors. https://aclanthology.org/2023.findings-acl.847/

Next: Knowing When Not to Answer

The safe-but-useless model exposes one more ambiguity.

Suppose the model does not adapt strongly to the context.

Why?

Possibility one:

the model ignored decisive facts

But there is another possibility:

the evidence genuinely does not justify a decisive answer

In that case, refusing to make a sharp recommendation may be correct.

We therefore need another question:

Does the system have enough evidence to answer at all?

That is not containment. It is not consistency. It is not sensitivity.

It is epistemic adequacy.

Chapter 11 turns abstention from an embarrassing non-answer into a measurable system capability.