Part I — A Vector Is Not Meaning

Two vectors, four answers

Here are two document vectors (3D, for legibility):

x = ( 2.0, 0.0, 0.0 )      a short doc, one strong topic
y = ( 6.0, 0.1, 0.0 )      a long doc, same topic, more of it
z = ( 0.0, 2.0, 0.0 )      a short doc, different topic

Ask “is x more similar to y or to z?” and the metric answers for you:

                  x–y        x–z
dot product       12.0        0.0      → y, overwhelmingly
euclidean dist     4.0        2.83     → z is CLOSER
cosine sim        ~1.00       0.0      → y, perfectly aligned

Euclidean distance says x and z are more similar, because x and y differ a lot in magnitude. Dot product and cosine say x and y, because they point the same way. Nothing about the documents changed. The decision rule changed.

“Similar” is not a property two texts have. It is the output of a metric applied to a representation. Choose both deliberately.

The metrics, derived

Dot product. x · y = Σ xᵢyᵢ = ‖x‖‖y‖cos θ. Rewards alignment and magnitude. If your model puts more-confident or more-specific content at larger norm, dot product will surface it — sometimes helpfully, sometimes as a length bias.

Cosine similarity. cos θ = (x · y) / (‖x‖‖y‖). Dot product with magnitude divided out. Pure orientation. Equivalent to the dot product of L2-normalized vectors, and equivalent to a monotone function of Euclidean distance between normalized vectors: ‖x̂ − ŷ‖² = 2 − 2cos θ.

Euclidean (L2) distance. ‖x − y‖ = √Σ(xᵢ − yᵢ)². Straight-line distance. Sensitive to magnitude. On normalized vectors it is just re-scaled cosine; on raw vectors it is a different ranking.

Manhattan (L1) distance. Σ|xᵢ − yᵢ|. Sums per-coordinate differences without squaring, so it down-weights single large deviations and is sometimes more robust in high dimensions.

The practical hierarchy:

    flowchart TD
    N{"normalize the vectors?"}
    N -->|no| RAW["dot product (magnitude matters) or L2 (magnitude matters, differently) — the rankings differ"]
    N -->|yes| UNIT["on the unit sphere: cosine == normalized dot == monotone in normalized L2 — the common default"]
  

Normalization is a modeling choice, not a formality

L2-normalizing every vector projects the space onto the unit sphere. This:

• Removes magnitude information. If norm encoded frequency, confidence, or document length, that signal is now gone — usually what you want for semantic search, sometimes not.
• Makes cosine, dot, and L2 rank-equivalent. One less thing to get wrong.
• Changes what “average” means. The mean of normalized vectors is not normalized; centroid-based methods (clustering, some retrieval) behave differently.

Many models are trained with normalized vectors and a temperature-scaled cosine objective. Using dot product on those at query time is a quiet mismatch.

Demonstration: the metric changes the winner on RELATE

MEASURED on RELATE v0.1, Wave 1 row 1.3 — artifact experiments/embeddings-from-first-principles/wave1/artifacts/metric-sweep.json. Model: all-mpnet-base-v2.

Rank the RELATE candidate pool per query under each metric, raw and normalized, and score with nDCG@10:

condition                         nDCG@10 (all)   nDCG@10 (hard-negative subset)
cosine  (raw = normalized)            0.9518              0.9577
dot     (raw = normalized)            0.9518              0.9577
euclidean (raw = normalized)         0.9518              0.9577
manhattan (raw = normalized)         0.9523              0.9590

MEASURED — the “several points” effect did not reproduce. The evaluated sentence-transformer models emit L2-normalized vectors, so “raw” and “normalized” are the same numbers, and on the unit sphere cosine, dot, and Euclidean rank identically (Manhattan is a hair different). The spread across all eight conditions is 0.0007 nDCG@10. The length-bias story is real, but only for embeddings that keep their magnitude — a raw-output encoder, a bag-of-words TF-IDF vector, a concatenated feature vector. For a modern normalized encoder, metric choice is close to inert; the decision that matters is whether to normalize at all, and these models made it for you. On hard negatives no metric helps, as before — but here that is because there is nothing left to choose.

What this chapter establishes and what it does not

Establishes: the four metrics and their algebraic relationships; that normalization is a modeling decision with consequences for ranking, averaging, and train/serve consistency; that metric choice materially affects ranking only when the embeddings retain magnitude — for an encoder that already L2-normalizes its output (measured on RELATE, row 1.3: spread 0.0007 nDCG@10 across eight conditions) the choice is close to inert — and that no metric can recover a distinction the representation did not capture.

Does not establish: a single best metric (it depends on how the model was trained and whether magnitude is signal), or that cosine is always right — it is a good default, not a law.

Lab 4: metric sweep

PROPOSED, not executed.

Setup. 200 labeled query–candidate pairs. One model. Compute embeddings once.

Task.

1. Rank under dot, cosine, L2, L1, each raw and L2-normalized (8 conditions).
2. Score each with Recall@1, Recall@10, MRR, nDCG@10.
3. Inspect the 10 pairs where raw-dot and cosine disagree most — what do they have in common?

Success criterion. State which metric your model wants (hint: check whether it was trained with normalized vectors) and quantify how much of your score is “metric” versus “representation” by comparing the best metric on all pairs to the best metric on hard negatives.

Companion component: the metric declaration

The Observatory never stores a similarity number without the rule that produced it:

similarity_spec:
  metric:        <dot | cosine | l2 | l1>
  normalization: <none | l2 | whitened>
  train_time_metric: <what the model was trained with, if known>
  mismatch_flag: <true if serve metric != train metric>

A stored score of 0.83 with no similarity_spec is uninterpretable and the Observatory refuses to compare it across spaces.

Failure modes

• Train/serve metric mismatch. Model trained with normalized cosine, queried with raw dot product. Silent quality loss.
• Reporting raw distances across corpora. A raw L2 of 4.0 is not comparable between a space of short titles and a space of long articles.
• Assuming cosine removes all nuisance. It removes magnitude. Anisotropy, hubness, and frequency effects survive (Chapters 6–8).
• Believing a better metric will fix hard cases. It will not. That is a representation problem (Chapters 11, 15).

What this chapter established

• Dot, cosine, L2, L1 — their formulas and their relationships (cosine = normalized dot = monotone in normalized L2).
• Normalization is a modeling decision affecting ranking, averaging, and train/serve consistency.
• On RELATE v0.1 with a normalized encoder (row 1.3), metric choice moved nDCG@10 by <0.001 — cosine, dot, and Euclidean rank identically on the unit sphere. The length-bias effect needs magnitude-carrying embeddings; then it is real.
• The metric declaration: no similarity number is stored or compared without the rule that produced it.

Next

Part I treated the space as a whole. Part II goes inside it. The next chapter asks what an individual dimension means — dimension 173 — and shows the answer is usually “nothing on its own,” which forces a rethink of what the coordinates are for.