Retrieval Is a Policy

Mon, 07 Sep 2026 11:30:00 +0000

Part III — Retrieval Is an Experiment

“Just retrieve the relevant documents”

There is no such operation. What actually runs is a pipeline, and every stage has parameters that change the output:

    flowchart TD
    Q[query] --> QT["query transform — raw / rewrite / expand / HyDE / decompose"]
    QT --> RE["representation — which model, normalization, chunking"]
    RE --> CG["candidate generation — ANN index, nprobe / ef, pre-filters"]
    CG --> SI["similarity — metric, hybrid dense+sparse weighting"]
    SI --> TH["threshold — minimum score to survive"]
    TH --> RK["ranking — rerank top-N with a cross-encoder?"]
    RK --> DV["diversity / dedup — MMR, cluster collapse"]
    DV --> TK["top-k — how many survive"]
    TK --> CA["context assembly — order, dedup, budget, citations"]
    CA --> M["what the model actually sees = the system's memory"]

Retrieval does not return “the relevant documents.” It returns whatever this specific chain of decisions produces — and that is the system’s memory.

Building an Embedding Runtime

Mon, 07 Sep 2026 16:00:00 +0000

Part VIII — Embeddings Become Infrastructure

Not a RAG chatbot

The obvious capstone would be “build a RAG system.” It would also be the wrong one — a RAG system exercises maybe a third of this book and hides the rest inside a framework.

The capstone is an Embedding Observatory: a runtime whose job is to know things about representations. It ingests text, documents, and queries under one or more embedding models, and it exposes not just vectors but the geometry, neighborhoods, distributions, evaluations, calibrations, versions, and bridges that the book spent 23 chapters building tools for.

System Design on Programmer.ie: Modern AI programming

Retrieval Is a Policy

“Just retrieve the relevant documents”

Building an Embedding Runtime

Not a RAG chatbot