Retrieval Is a Policy

Mon, 07 Sep 2026 11:30:00 +0000

Part III — Retrieval Is an Experiment

“Just retrieve the relevant documents”

There is no such operation. What actually runs is a pipeline, and every stage has parameters that change the output:

    flowchart TD
    Q[query] --> QT["query transform — raw / rewrite / expand / HyDE / decompose"]
    QT --> RE["representation — which model, normalization, chunking"]
    RE --> CG["candidate generation — ANN index, nprobe / ef, pre-filters"]
    CG --> SI["similarity — metric, hybrid dense+sparse weighting"]
    SI --> TH["threshold — minimum score to survive"]
    TH --> RK["ranking — rerank top-N with a cross-encoder?"]
    RK --> DV["diversity / dedup — MMR, cluster collapse"]
    DV --> TK["top-k — how many survive"]
    TK --> CA["context assembly — order, dedup, budget, citations"]
    CA --> M["what the model actually sees = the system's memory"]

Retrieval does not return “the relevant documents.” It returns whatever this specific chain of decisions produces — and that is the system’s memory.

Reranking on Programmer.ie: Modern AI programming

Retrieval Is a Policy

“Just retrieve the relevant documents”