Your Model Is a Dependency
Your Model Is a Dependency
In chapter 5 we composed several LM operations into one small editorial program. It can analyze a sentence, rewrite it, assess risk, and return structured fields.
One assumption remains hidden:
Which model executed the program?
That question is not operational trivia. A language-model program is not fully described by its Python class and signature. Its behavior depends on the LM boundary.
DSPy program
↓
LM boundary
/ \
local hosted
model model
This chapter treats the model as a dependency: configured, recorded, tested, and replaced deliberately.
1. Configure the LM explicitly
In current DSPy, the common configuration shape is:
Then the same program can run:
program = EditorialRewriteProgram()
result = program(
sentence="She opened the door and then she laughed.",
editorial_goal="Sharpen rhythm without changing the event.",
local_context="The scene is tense but not comic.",
)
For local models, DSPy commonly routes through provider integrations such as LiteLLM-compatible model strings. The exact string depends on the installed DSPy/LiteLLM version and local provider support. A typical Ollama-style configuration may look like this in environments that support it:
lm = dspy.LM(
"ollama_chat/qwen3",
api_base="http://localhost:11434",
api_key="",
)
dspy.configure(lm=lm)
Do not treat that snippet as a universal promise. Treat the LM boundary as something to verify and record in your environment:
2. What should change when the model changes?
Ideally, the task contract should remain stable:
sentence
editorial_goal
local_context
↓
rewritten_text
rationale
confidence
The implementation may need adaptation:
| Dependency change | Possible effect |
|---|---|
| Hosted to local | Different latency, capacity, availability, privacy, and output behavior |
| Larger to smaller model | Different instruction following, schema reliability, latency, and cost |
| Temperature / sampling change | Different output distribution and run-to-run variation |
| Provider change | Different timeouts, errors, rate limits, supported features, and response behavior |
| Context limit change | Different truncation or context-selection requirements |
| Adapter change | Different request formatting and response parsing |
The right response is not to quietly rewrite the signature for each model. First hold the contract steady and observe which failure mode appears.
same program
same cases
different LM dependency
↓
behavior comparison
Only after that comparison should you decide whether the contract, module, validation, or model needs to change.
3. Silent fallback is dangerous
Writer contains a useful production lesson here. Its DSPy/Ollama sentence-repair path records whether a generation used a real LLM. DTOs include used_real_llm, llm_call_count, provider, model, API base, elapsed time, warnings, and candidate metadata. Tests assert that a deterministic fallback cannot masquerade as a DSPy/Ollama result.
That distinction matters because fallback output can be useful operationally but invalid as LM evidence.
real DSPy/Ollama response
↓
eligible as model behavior evidence
deterministic fallback
↓
useful degraded behavior
↓
not evidence that the LM performed the task
A bad fallback design hides infrastructure failure:
try:
return program(**inputs)
except Exception:
return {"rewritten_text": sentence, "confidence": 0.0}
That keeps the UI alive, but it corrupts evaluation if the result is recorded as a model output.
A better shape records the state and falls back only for errors that the LM boundary has classified as dependency failures:
4. Reproducibility needs model identity
If a run matters, store enough information to explain it later:
CoCoder’s DSPy experiment protocol freezes this kind of context for optimization: program and baseline version, dataset fingerprints and splits, runtime/provider identity, validation contract, comparison policy, optimizer configuration, DSPy version, seed, budget, and source revision. The point is not bureaucracy. The point is that changing any of those fields changes what the evidence means.
For our small program, a simple run record is enough:
import hashlib
import json
from datetime import datetime, timezone
def fingerprint(payload: dict) -> str:
blob = json.dumps(payload, sort_keys=True, default=str)
return hashlib.sha256(blob.encode("utf-8")).hexdigest()
def make_run_record(
inputs: dict,
output: ProgramRunResult,
*,
lm_config: dict,
dspy_version: str,
adapter_id: str,
) -> dict:
return {
"created_at": datetime.now(timezone.utc).isoformat(),
"program": "editorial_rewrite_program",
"program_version": "0.1",
"dspy_version": dspy_version,
"lm_provider": output.lm_provider,
"lm_model": output.lm_model,
"lm_config_fingerprint": fingerprint(lm_config),
"adapter_id": adapter_id,
"used_real_lm": output.used_real_lm,
"fallback_state": output.fallback_state,
"input_fingerprint": fingerprint(inputs),
"output_fingerprint": fingerprint(output.output),
"error": output.error,
}
This is not optimizer provenance yet. That comes later. For now, the important discipline is that model identity alone is not enough: a run record should identify the provider-facing configuration and DSPy/adapter boundary well enough to explain which execution environment produced the output.
---
## 5. Debug the LM boundary first
Some apparent program failures originate at the LM/provider boundary rather than in the signature or module.
| Symptom | Likely cause | How to diagnose it | What to change |
| --- | --- | --- | --- |
| Every output is empty | Provider call failed or wrong model name | Run a health check directly against the provider | Fix model identifier/base URL |
| Outputs ignore field structure | Model, provider feature support, or adapter/parsing mismatch | Inspect raw LM history plus adapter/parse warnings | Verify adapter/provider support before redesigning the task |
| Local model works for direct prediction but not composed program | Context/token budget or instruction-following limit | Compare stage-by-stage requests and outputs | Reduce context/stages or choose a model with suitable capacity |
| Repeated runs behave differently from expectation | Sampling, rollout, or cache assumptions differ | Record LM kwargs, rollout IDs, and cache policy | Make the intended stochastic/cache behavior explicit |
| Fallback results appear in evaluation | Fallback state is not filtered | Count `used_real_lm == false` records | Exclude fallback from model-quality claims |
Before redesigning the program, confirm that the dependency is healthy. A broken endpoint can look like a bad signature. A context-limit failure can look like poor reasoning. A silent fallback can look like a robust model.
---
## 6. Hosted and local are engineering choices
A local model is not automatically more serious because it is local. A hosted model is not automatically better because it is larger. They expose different constraints.
| Choice | Advantages | Costs |
| --- | --- | --- |
| Local Ollama | Control, privacy, offline development, direct control over the serving environment | Local hardware, model capacity, setup, and structured-output behavior become your responsibility |
| Hosted API | Access to managed infrastructure and a wide range of model capabilities | Cost, rate limits, external dependency, and possible model/version drift |
| OpenAI-compatible gateway | Routing flexibility and centralized policy | Another dependency layer whose configuration and failures must be observed |
DSPy helps because the program can often remain stable while the LM dependency changes. But "often" is doing work. You still need to test the same program under the target model.
---
## Conclusion
We gained an explicit LM boundary. The model is now a dependency of the program, not an invisible assumption.
We removed the assumption that a DSPy program is fully described by its signatures and modules alone. A useful run record must also identify the LM/provider configuration and the surrounding DSPy execution boundary closely enough that later comparisons have interpretable provenance.
What remains unsolved is improvement. We can now specify and execute a language-model program across model backends. But we still have no disciplined way to teach it from examples or determine whether one version is better.
That is the transition into chapter 7:
> Examples are data, not decoration.
