Build Programs From Programs
Build Programs From Programs
In chapter 4 we ran the same signature with different execution policies. That helped isolate one question:
Should this behavior be attempted directly or with intermediate reasoning?
Real applications usually ask a larger question. A sentence-improvement system may need to find what is wrong, generate a candidate, assess risk, and return something the rest of the product can store.
That is where a single LM call starts to hide several distinct operations.
input
↓
analyze
↓
rewrite
↓
assess
↓
return
This chapter builds a small composed DSPy program. It is still much smaller than Writer’s production sentence workflows or CoCoder’s repository repair programs, but it introduces the same design pressure: make intermediate state explicit enough to inspect.
1. The giant prompt version
We could put everything into one prompt:
Analyze this sentence, identify issues, rewrite it, assess risk,
explain your decision, and return JSON...
That sometimes works. It is also hard to debug.
If the final rewrite is bad, which part failed?
context interpretation?
issue diagnosis?
rewrite generation?
risk assessment?
output formatting?
A giant prompt compresses multiple operations into one opaque step. Decomposition gives us places to inspect behavior. It also adds complexity, so we should decompose only where the intermediate state is useful.
2. Three small contracts
First, define separate signatures for analysis, rewriting, and risk assessment.
These are not three random prompts. They are three program boundaries.
One naming choice matters: preservation_notes are model-inferred intermediate state, not hard policy. If the application has a true constraint such as “the character name Jalen must not change,” pass that requirement explicitly or enforce it deterministically. Do not let an analysis model invent the hard constraints that later stages are judged against.
3. Compose them in a custom module
A DSPy module can contain submodules and ordinary Python.
Two details are deliberate. First, normalize_preservation_notes is deterministic Python. Normalizing a list shape does not require another LM call.
Second, the custom module does not automatically copy the reasoning field produced internally by ChainOfThought into its application-facing result. The module decides which state crosses its boundary. If reasoning is useful for debugging or optimization, record it deliberately rather than assuming every downstream caller should receive it.
Not everything in a DSPy program should be an LM call, and not every internal field should become part of the public result.
4. Deterministic code belongs in the program too
Some tasks are better handled by ordinary software:
Writer’s DSPy sentence-improvement engine follows this pattern. It uses deterministic sentence splitting, reason-code heuristics, evidence packet hashes, changed-token ratios, and provenance DTOs around the LM generation provider. The LM creates candidates. The runtime records and structures evidence. It does not mutate chapter files.
That separation is essential:
5. Make intermediate representations inspectable
The composed program now produces more than a final sentence:
CoCoder’s EngineeringProgram architecture is a larger version of the same idea. Candidate patch generation, validation evidence, optimizer score, promotion recommendation, and program version are separate concepts. That separation prevents an optimizer from becoming the authority on whether its own output should be deployed.
6. The complexity tax
Decomposition is not free.
| Benefit | Cost |
|---|---|
| Easier failure localization | More calls and more latency |
| Inspectable intermediate state | More schemas and traces to maintain |
| Reusable stages | More integration failures |
| Narrower optimization targets | More ways to contaminate experiments |
| Stage specialization | Upstream model errors can propagate downstream |
For a one-sentence rewrite, three LM calls may be excessive. A direct Predict module may be better. The composed version earns its keep only if intermediate state changes what the program can observe, test, reuse, decide, or optimize.
Use this rule:
Split a program when the intermediate result changes what you can observe, test, reuse, or decide.
What Usually Goes Wrong
| Symptom | Likely cause | How to diagnose it | What to change |
|---|---|---|---|
| The composed program is slower but not better | Decomposition added calls without useful state | Compare direct and composed versions on the same cases | Collapse unnecessary stages |
| Analysis invents a “constraint” that later stages obey | Model-inferred state was mistaken for hard policy | Compare intermediate notes with explicit task inputs and deterministic rules | Name inferred state honestly; keep hard constraints explicit |
| Later stages ignore earlier outputs | Intermediate fields are vague or unenforced | Log each stage and inspect contradictions | Make intermediate fields narrower or add deterministic checks |
| Debug logs are huge | The program records raw everything but no summary | Inspect stored run payloads | Store bounded traces, fingerprints, and key fields |
| Every step is an LM call | Deterministic work was delegated to the model | Search for LM calls that ask computable questions | Move checks into Python |
Conclusion
We gained a small but nontrivial language-model program. It is made of smaller programs plus deterministic code. It has intermediate state that can be inspected when behavior fails.
We removed the assumption that an LM application must be either one prompt or a large agent framework.
The remaining assumption is model-related. Our program currently behaves as though “the model” is a background detail. It is not.
Next we ask:
Is the model part of the program or a dependency of it?
