DSPy From First Principles

A language-model application does not have to remain a collection of prompts.

It can become a program.

That program can have explicit inputs and outputs. It can be composed from smaller components. It can run against different language models. It can be evaluated on examples, optimized against a metric, versioned as an artifact, compared against a baseline, and promoted only when the evidence justifies the change.

This book builds that system from first principles.

We begin with an ordinary handwritten prompt and keep asking what is missing. Each answer introduces another mechanism: a contract, a module, a dataset, a metric, an optimizer, a tool boundary, a context strategy, an experimental firewall, a promotion policy.

The aim is not to memorize the DSPy API.

It is to understand what DSPy makes possible well enough that optimizers, agents, reflective search, and compiled programs stop looking like framework magic and become inspectable engineering machinery.

The recurring question throughout the book is:

What exactly is the program, what evidence says it is better, what was the optimizer allowed to see, and what justifies replacing the version that already works?

The program we will build

The progression begins simply:

task
  ↓
Signature
  ↓
Module
  ↓
language model
  ↓
structured output

Then we add the machinery required to improve that program empirically:

program
   +
examples
   +
metric
   ↓
baseline evaluation
   ↓
optimizer
   ↓
candidate program
   ↓
untouched evaluation
   ↓
comparison
   ↓
promote / reject

By the end of the book, that small program has grown into something much more substantial:

                        task
                         |
                         v
                  program contract
                         |
                         v
                execution strategy
                         |
              +----------+----------+
              |                     |
              v                     v
          LM modules          deterministic code
              |                     |
              +----------+----------+
                         |
                         v
                  tools + context
                         |
                         v
                     behavior
                         |
                         v
                    observation
                         |
                         v
                     metric
                         |
                  score + feedback
                         |
                         v
                    optimizer
                         |
                         v
                candidate program
                         |
                         v
               independent holdout
                         |
                         v
                    comparison
                         |
                         v
               promote / reject
                         |
                         v
                 production evidence
                         |
                         v
                 next experiment

A language model still performs some of the work.

But it is no longer the whole system.

The rest is software, data, evaluation, experimental discipline, and evidence.

That distinction is the foundation of this book.

What this book is designed to teach

By working through the chapters, you will learn how to turn the hidden parts of an LLM application into explicit engineering objects.

You will learn how to:

• distinguish a handwritten prompt from a language-model program;

• express a task as a semantic contract rather than burying the interface inside prose;

• use DSPy Signatures to define what a component consumes and produces;

• separate what a program should do from how it attempts to do it;

• compare execution strategies such as direct prediction and explicit reasoning under the same contract;

• compose several LM operations into one inspectable program;

• decide which parts of the system belong in deterministic Python rather than in another model call;

• treat the underlying language model as a configurable dependency instead of an invisible assumption;

• record provider, model, configuration, fallback state, and program identity as part of the evidence for a run;

• turn examples into structured data rather than copying demonstrations into prompts by hand;

• distinguish program inputs, labels, metadata, training examples, demonstrations, development cases, and holdout evidence;

• establish a baseline before allowing an optimizer to change anything;

• design metrics that reflect the task rather than merely rewarding convenient proxies;

• attack metrics with adversarial examples before trusting an optimizer to pursue them;

• use deterministic constraints, reference evidence, LM judges, and human preference without confusing one form of evidence for another;

• understand DSPy compilation as candidate-program generation rather than mysterious prompt improvement;

• use few-shot optimization to construct or select demonstrations;

• use instruction optimization to search over LM-facing program state;

• use reflective feedback to explain failures and propose better instructions;

• understand why reflection proposes changes while evaluation still decides whether those changes helped;

• treat tool-using agents as language-model programs whose trajectories, tools, and outcomes can also be measured and optimized;

• design bounded tool interfaces instead of granting an LM uncontrolled access to its environment;

• distinguish direct context, retrieval, memory, agent-directed search, and programmatic exploration of large contexts;

• prevent training labels, future repository revisions, solution patches, evaluator outputs, or repeated holdout inspection from contaminating an experiment;

• version programs, datasets, metrics, providers, tool surfaces, retrieval systems, and validation policies so an experimental result remains interpretable;

• distinguish an optimizer score from independent evidence that a candidate is actually better;

• separate evaluation, comparison, promotion, activation, observation, rejection, and rollback;

• treat production activity as a source of future evidence rather than automatically converting every interaction into a training label; and

• assemble the mechanisms into a repository-repair program that can improve through controlled experiments without silently promoting its own changes.

The objective is not to make prompt optimization more elaborate.

It is to make language-model behaviour more understandable, measurable, reproducible, and improvable.

One program, built progressively

The chapters describe one growing engineering system.

01  Why Are We Still Hand-Writing Prompts?

    Start with a reasonable prompt and discover why a string
    becomes an inadequate software abstraction.


02  A Prompt Is Not Yet a Program

    Give language-model behaviour a program boundary:
    inputs, declared behaviour, execution, and outputs.


03  Define the Contract

    Use Signatures to express what the program consumes
    and what downstream software can rely upon.


04  Separate What From How

    Keep the task contract stable while changing the
    execution strategy used to satisfy it.


05  Build Programs From Programs

    Compose smaller LM components with deterministic
    Python into an inspectable multi-stage program.


06  Your Model Is a Dependency

    Make provider, model, configuration, failure,
    and fallback behaviour explicit.


07  Examples Are Data, Not Decoration

    Turn examples into persistent data with identities,
    provenance, inputs, labels, and frozen splits.


08  You Cannot Optimize What You Cannot Measure

    Establish a named baseline and record per-case
    evidence before attempting optimization.


09  The Metric Is Part of the Program

    Define what improvement means, combine hard and
    soft signals, and attack the metric before trusting it.


10  Compile the Program

    Treat optimization as the production of a candidate
    program under a frozen experimental boundary.


11  Few-Shot Optimization

    Let the optimizer select and construct demonstrations
    instead of hand-pasting examples into prompts.


12  Optimize the Instructions

    Search instruction and demonstration space under
    an explicit metric, development set, and budget.


13  Let the Program Reflect

    Turn scalar failure into diagnostic feedback that
    can guide reflective program improvement.


14  Agents Are Programs Too

    Give the program tools and let it act on an environment
    without abandoning contracts, measurement, or validation.


15  Search, Memory and Long Context

    Decide what evidence the program should inspect when
    the relevant environment is larger than the prompt.


16  Don't Let the Optimizer Cheat

    Build an experimental firewall around holdouts,
    tools, memory, retrieval, history, and evaluation.


17  From Experiment to Production

    Compare candidates independently, promote deliberately,
    observe active versions, and retain the ability to rollback.


18  Build a Self-Improving Engineering Program

    Assemble the mechanisms into a repository-repair
    system whose improvement claims are backed by evidence.

Each chapter adds a mechanism because the previous system has exposed a specific limitation.

That progression matters.

An optimizer makes little sense before we have a metric. A metric is difficult to trust before we have examples and a baseline. Holdout isolation becomes much more important once an optimizer can search instructions, retrieve memories, call tools, inspect repository history, and learn from diagnostic feedback.

The final chapter therefore does not introduce another optimizer.

It asks whether everything we have already built can actually compose.

The resulting system looks roughly like this:

engineering request
        ↓
freeze repository revision
        ↓
inspect available evidence
        ↓
search / retrieve / use tools
        ↓
diagnose
        ↓
propose intervention
        ↓
generate candidate patch
        ↓
apply in isolation
        ↓
validate
        ↓
inspect failures
        ↓
revise when justified
        ↓
record outcome evidence
        ↓
evaluate program behaviour
        ↓
build audited dataset
        ↓
run controlled optimization
        ↓
candidate program version
        ↓
independent holdout evaluation
        ↓
compare against active baseline
        ↓
PROMOTE / REJECT / INSUFFICIENT
        ↓
operator decision
        ↓
observe production

The point is not autonomous self-modification.

It is controlled empirical improvement.

The central engineering principle

Language models are probabilistic.

That is useful.

They can generate rewrites, diagnoses, plans, patches, search queries, tool selections, explanations, hypotheses, and alternatives that would be difficult to encode as ordinary deterministic logic.

But not every part of an LLM application should therefore become probabilistic.

When ordinary software can establish something reliably, let ordinary software establish it.

Does this output parse?                 → parser

Does it satisfy the schema?             → validator

Is this tool allowed?                   → capability policy

Which examples belong to holdout?       → dataset manifest

Did the patch apply?                     → patch engine

Did the test pass?                       → test runner

Did the named entity survive?           → deterministic check

Which program version generated this?   → artifact registry

Did the optimizer see this case?        → experiment audit

Did the candidate beat the baseline?    → evaluator + comparison policy

Which version is active?                → deployment registry

That gives us one of the book’s central rules:

Use the language model where uncertainty and generative reasoning are useful. Move correctness, identity, boundaries, measurement, and governance into inspectable software wherever possible.

DSPy then occupies a very specific place.

It gives us machinery for expressing and improving the probabilistic parts of the program.

It does not remove the need for the deterministic system around them.

Optimization is not promotion

One distinction will recur throughout the book:

optimizer
   ↓
candidate

evaluation
   ↓
evidence

comparison
   ↓
recommendation

promotion
   ↓
active version

Those are different operations.

An optimizer may discover a program that performs better on development examples.

That does not entitle the candidate to replace the active program.

A reflective optimizer may explain why an example failed.

That does not make its proposed fix correct.

An agent may produce a convincing trajectory.

That does not prove that the requested outcome occurred.

A compiled program may achieve a higher scalar score.

That does not erase a new hard regression.

The book therefore keeps returning to another rule:

The optimizer proposes. Independent evaluation establishes evidence. Promotion is a separate decision.

That separation becomes more important as the system becomes more capable.

Evidence has a boundary

Optimization introduces another problem: the system can learn the experiment instead of learning the task.

A program may accidentally receive a human decision as an input.

A demonstration may contain a holdout case.

A retrieval index may contain the eventual solution.

A repository agent may inspect a later commit containing the exact patch.

Reflective feedback may reveal the gold answer.

A developer may inspect holdout failures, modify the program, and then call the same cases an untouched test set.

All of these can create impressive-looking results.

None provide trustworthy evidence of general improvement.

The experimental boundary therefore becomes part of the software:

allowed:

training examples
development examples
decision-time repository evidence
approved tools
approved memory
approved retrieval sources


not allowed:

holdout labels
future outcomes
solution patches
post-decision metadata
hidden evaluator state
future repository revisions

By the end of the book, a program version is meaningful only together with the evidence describing how it was created and evaluated.

What you should be able to do after reading

The goal is not that you memorize every DSPy optimizer or every current API.

DSPy will change.

The more durable goal is that unfamiliar DSPy and LLM-programming systems become legible.

You should be able to open one and ask:

What is the actual program?

What is its task contract?

Which parts are LM behaviour?

Which parts are deterministic software?

What execution strategy is being used?

Which model and provider executed it?

What examples can the program see?

What examples can the optimizer see?

Which cases are genuinely untouched?

What does the metric reward?

Can the metric be gamed?

What can the optimizer change?

What artifact did optimization produce?

What feedback caused a reflective mutation?

What tools are available?

What evidence can retrieval or memory expose?

Could any of those sources leak the answer?

How is a candidate evaluated?

Under what conditions is it compared with the baseline?

Who decides whether it is promoted?

Which version is active?

Can it be rolled back?

What production evidence becomes eligible for the next experiment?

Those questions are more durable than the current name of an optimizer.

They also make debugging much more precise.

Instead of saying:

DSPy did not improve my program.

we can ask whether the failure occurred in the contract, examples, split, metric, model, demonstrations, instruction search, reflection feedback, retrieval, tool use, evaluation protocol, or promotion policy.

That is a much more useful way to engineer language-model systems.

You will also learn when not to optimize

DSPy makes optimization available.

That does not mean every program should be optimized.

If a direct Predict reliably solves the task, adding a multi-stage module may only add latency and failure modes.

If deterministic code can enforce a constraint, searching for an instruction that persuades the model to follow it may be the weaker design.

If the dataset is tiny or noisy, an optimizer may mostly learn accidents.

If the metric does not represent the real objective, better optimization can make the system worse faster.

If a retrieval failure is causing the model to miss the necessary evidence, instruction optimization may solve the wrong problem.

If an existing program already satisfies the real requirement, searching for a numerically better one may have no practical value.

Throughout the book we therefore start from a baseline and add machinery only when a specific failure justifies it.

The aim is not maximum optimization.

It is useful improvement that survives measurement.

Who this book is for

You should be comfortable with Python and with the basic idea of calling a language model.

You do not need prior DSPy experience.

You do not need to know its optimizer APIs in advance.

You also do not need to have built a full agent system.

The examples begin with ordinary Python, explicit data structures, small DSPy programs, and visible evaluation code. More advanced mechanisms arrive only after the pieces they depend upon have been established.

If you have already built production LLM applications, the later chapters are intended to be equally useful.

They focus increasingly on the problems that appear after the first demo works:

How do we know this version is better?

What exactly changed?

Can we reproduce it?

Did the optimizer cheat?

Can the program act safely?

Can it find the right context?

What evidence earns deployment?

What happens when production proves us wrong?

Those are engineering questions rather than prompting questions.

What this book does not try to cover

This is not an encyclopedia of every DSPy class.

It does not attempt to document every optimizer option, adapter, provider, experimental module, deployment integration, or API parameter.

Those details change too quickly to form the conceptual spine of the book.

Instead, the book focuses on the mechanisms underneath them:

contracts
execution
composition
data
measurement
optimization
reflection
tools
context
experimental isolation
versioning
comparison
promotion
evidence

We will use current DSPy APIs to make those mechanisms concrete.

But the aim is that the reasoning survives an API change.

This is also not a claim that every language-model application should become a self-improving system.

Sometimes the right solution is one prompt.

Sometimes it is one Predict.

Sometimes it is ordinary Python.

Sometimes it is a fixed workflow.

The machinery in this book should be earned by a problem it solves.

The promise

By the end of DSPy From First Principles, you should be able to look at an unfamiliar language-model application and work out:

what the program actually is, what behaviour it specifies, what evidence measures that behaviour, what the optimizer is allowed to change, what information it was allowed to see, and what evidence would justify replacing the current version.

You should also be able to ask one final question:

When this system says the new program is better, what exactly makes that claim trustworthy?

Once you can answer that question, DSPy stops being a collection of prompt-optimization techniques.

It becomes something much more useful:

a way to make probabilistic language-model behaviour part of an empirical engineering process.