Part III — Retrieval Is an Experiment

Retrieval in four lines

query_vec = embed(query)                       # one vector
scores    = corpus_vecs @ query_vec            # one dot product per document
order     = np.argsort(-scores)                # sort, descending
results   = [corpus[i] for i in order[:k]]     # take the top k

That is the entire primitive. Everything a vector database adds — indexes, quantization, sharding, filtering — is an optimization or an operational convenience around these four lines. Understand them completely before adding anything.

What exactly is a retrieval system deciding, and which of its parameters change the answer rather than just the speed?

    flowchart TD
    Q[query] --> QT["query construction — raw / expand / HyDE / multi-query (changes the query vector)"]
    QT --> E["embed — model + normalization (dominates everything below)"]
    E --> SC["score every doc — metric: dot / cosine / L2"]
    SC --> SR["sort descending, cut at k, drop below threshold"]
    SR --> PP["de-duplicate / diversify (MMR); apply the metadata filter pre- or post-search"]
    PP --> R[results]
    SC --> ANN["ANN index (HNSW / IVF / PQ) changes SPEED, not the intended answer — but measure its recall against exact brute force"]
  

The parameters that change the answer

• The representation. Which model, which normalization. Established in Parts I–II. This dominates everything below.
• The metric. Dot vs. cosine vs. L2 (Chapter 4).
• k. How many results. Not a display choice — it sets recall/precision trade-off and, downstream, how much context a model sees.
• The threshold. A minimum score to be returned at all. Without one, you always return k results even when nothing is relevant (Chapter 10). With a bad one, you return nothing when something is (Chapter 14).
• Query construction. Raw query, expanded query, HyDE-style hypothetical answer, multi-query. Changes the query vector, changes everything.
• De-duplication / diversity. MMR and similar: trade some relevance for coverage.
• Filtering. Metadata pre-filters (date, source, permissions) applied before or after the vector search — and the two orders give different results.

The parameters that change only the speed (and a little accuracy)

• Exact vs. approximate search. Brute force is O(nd) per query and exact. Approximate nearest-neighbor (ANN) indexes trade a small, tunable recall loss for large speed gains.
• Index family.
  • IVF (inverted file): cluster the corpus, search only the nearest nprobe clusters. Miss rate depends on whether the true neighbor is in a searched cluster.
  • HNSW (hierarchical navigable small-world graph): a navigable graph; ef_search controls how hard it looks. The common default.
  • PQ / OPQ (product quantization): compress vectors into codes; approximate distances from the codes. Cuts memory drastically, adds quantization error.
  • ScaNN, DiskANN, etc.: further points on the speed/memory/recall surface.
• ef_search / nprobe / nlist. The knobs that trade recall for latency.

The critical point: when ANN error appears, it is not random. In a large index it systematically drops the hardest neighbors — the ones just barely closer than the runner-up — which are often exactly the semantically interesting ones. (On a small index it may not appear at all: RELATE v0.1’s 1,173 vectors are recovered exactly by HNSW even at ef_search=10, row 1.6.) Either way, measure ANN recall against the exact result, not against the labels.

De-duplication is not a nicety

Real corpora have near-duplicates: boilerplate, repeated passages, the same fact stated five ways. Naive top-k fills with them, and a model reading the context sees “five sources agree” when it is one source copied. Retrieval quality includes distinct coverage, not just relevance.

Demonstration: brute force vs. HNSW on RELATE

MEASURED on RELATE v0.1, Wave 1 row 1.6 — artifact experiments/embeddings-from-first-principles/wave1/artifacts/ann-vs-exact.json. Model all-mpnet-base-v2, hnswlib, cosine space, M=16.

Index all 1,173 RELATE items. Query with the 269 labeled queries. HNSW ef_search sweep, recall measured against exact brute force:

ef_search   Recall of exact top-10   Recall of graded-relevant items
10                 0.995                        1.00
20                 1.000                        1.00
40                 1.000                        1.00
80                 1.000                        1.00

MEASURED — the “systematic error” effect did not show up at this scale. A 1,173-vector HNSW graph reproduces the exact top-10 almost perfectly even at ef_search=10, and never drops a graded-relevant item. The claim that ANN error concentrates on the hardest neighbors is a real, documented property of large indexes (millions of vectors, where the graph is sparse relative to the data); RELATE is far too small to exhibit it. On a corpus this size brute force is already sub-millisecond — the index is solving a problem you may not have yet. The methodological point stands regardless: whatever index you use, measure its recall against the exact primitive, because when ANN error does appear it is not uniform.

What this chapter establishes and what it does not

Establishes: retrieval is embed → score → sort → cut; the parameters that change the answer (representation, metric, k, threshold, query construction, dedup, filter order) versus those that mostly change speed (exact/approximate, index family, its knobs); ANN error at scale is systematic, not random (though a small index like RELATE v0.1 shows none — row 1.6), and must always be measured against the exact result.

Does not establish: which index to use (workload-dependent), or that approximate search is bad (it is essential at scale). It establishes the order of operations: get the exact primitive right, measure, then optimize.

Lab 9: build retrieval, then approximate it

PROPOSED, not executed.

Setup. 1,000–10,000 items, 200+ labeled queries, one model.

Task.

1. Implement brute-force retrieval. Record Recall@{1,5,10}, MRR, latency.
2. Build an HNSW index. Sweep ef_search ∈ {16, 40, 100, 400}. Record recall vs. labels, recall vs. exact, latency.
3. Add PQ. Record the same.
4. For the best-speed config, list the 20 queries it misses relative to exact. Categorize them.

Success criterion. State the smallest corpus size at which brute force exceeds your latency budget, and the recall-vs-exact you are willing to trade there — as a decision, not a default.

Companion component: the retrieval spec

retrieval_spec:
  space_record:     <from Ch1>
  similarity_spec:  <from Ch4>
  k:                int
  threshold:        float | none   (see Ch14 for how it is set)
  query_transform:  <raw | expand | hyde | multi>
  dedup:            <none | mmr(lambda) | cluster>
  filter_order:     <pre | post>
  index:            <exact | hnsw(ef) | ivf(nprobe) | +pq(m)>
  ann_recall_vs_exact: float   (measured, not assumed)

The Observatory stores this with every retrieval run so a quality change can be traced to the parameter that caused it.

Failure modes

• Adding an index before measuring brute force. You may be optimizing a non-problem and inheriting systematic recall loss.
• Measuring ANN recall against labels only. Hides the index’s contribution; measure against the exact result.
• No threshold. Always returning k items guarantees irrelevant context when the corpus lacks an answer.
• Ignoring near-duplicates. “Multiple sources” that are one source, copied.
• Filter-order blindness. Pre-filter and post-filter give different results and different recall.

What this chapter established

• The four-line retrieval primitive and the discipline of understanding it before indexing.
• Answer-changing parameters vs. speed-changing parameters.
• ANN error, where it occurs, is systematic — it drops the hardest, most valuable neighbors — and must always be measured against the exact result (RELATE v0.1 is too small to show any — row 1.6).
• De-duplication and filter order are correctness concerns, not conveniences.
• The retrieval spec: every knob recorded so quality changes are attributable.

Next

Every retrieval run so far returned its top k and moved on. The next chapter asks whether the top result is right — and builds the cases where the nearest neighbor is fluent, on-topic, geometrically closest, and wrong.