Part VI — Crossing Embedding Spaces

A family of maps, ordered by how much they can bend the space

• The null map (T(x) = x). Do nothing. Always run this first. If it already scores well, your two spaces were nearly the same space to begin with (this happens for models of the same size, backbone, and objective) and any “translation” you fit is fitting noise. vec2vec’s own baseline is exactly this, and for their near-identical model pairs it reaches top-1 ≈ 1.0 — the real work is only on pairs where the null map collapses.
• Orthogonal Procrustes. T is constrained to a rotation/reflection: min ‖X_A R − X_B‖² subject to RᵀR = I. Solution: R = UVᵀ from the SVD of X_Aᵀ X_B. Preserves all distances and angles within space A — it only re-orients. Cannot fix scale or shear. Best when the two spaces are “the same shape, different pose.”
• Linear regression (least squares). Chapter 18. T is any linear map: rotation + scaling + shear + projection. More expressive; can overfit; does not preserve within-A geometry.
• CCA (Canonical Correlation Analysis). Finds paired directions in A and B that are maximally correlated; aligns in that shared subspace. Naturally handles different dimensions and discards directions with no counterpart. Good when the two spaces share a subspace but also have private structure. Note it maximizes correlation, which is not the same as preserving retrieval rank — highly correlated shared directions can still reorder near-ties.
• Relative representations (“re-express, don’t map”). Instead of fitting a map at all, replace every vector by its list of cosine similarities to a fixed set of shared anchor items (Moschella et al., 2023). That representation is invariant to rotation and rescaling by construction, so two spaces expressed this way become directly comparable — no T to fit. The cost: it needs a shared anchor set (texts embedded in both spaces — paired data by another name), and the invariance holds “under the same data and modeling choices.”
• Learned nonlinear map (small MLP). T is a 2–3 layer network. Most expressive; needs the most anchors; highest overfitting risk; hardest to reason about. Use only if the linear family measurably plateaus below your target.

The ordering is deliberate: try the most constrained map that meets your preservation target. A rotation that gets you to 0.85 retrieval agreement is better than an MLP that gets you to 0.87, because the rotation has fewer ways to be wrong on data you have not seen. The recent cross-encoder alignment work reinforces this: vec2vec’s nonlinear translator and mini-vec2vec’s linear one reach comparable coarse quality, and the linear one is far cheaper and more stable (Dar, 2025). Where nonlinearity earns its keep is not representational capacity but optimization robustness in the unpaired regime — when there are no anchors to regress on, the extra flexibility helps the alignment converge, not the final map express more.

Preprocessing that changes everything

• Mean-centering both spaces before alignment removes the offset direction (Chapter 8) and usually helps a lot.
• Per-space normalization (unit norm, or whitening) puts both on comparable scales; Procrustes in particular assumes comparable scale.
• Anchor selection. Anchors should span the domain and the difficulty range. All-easy anchors give a bridge that fails on hard cases. Include hard-negative-adjacent items.
• Dimension mismatch. Regression and CCA handle it directly. For Procrustes, project the larger space to the smaller with PCA first (and record the PCA matrix in the space identity).

Coordinate reconstruction vs. semantic preservation

Two different questions:

Coordinate reconstruction:   is  T(E_A(x))  numerically close to  E_B(x)?
                             measured by cosine / MSE to the target vector

Semantic preservation:       does  T(E_A(·))  support the same decisions
                             as  E_B(·)?
                             measured by neighbor overlap, rank correlation,
                             cluster agreement, retrieval metrics, threshold transfer

They come apart constantly:

• A map can have mediocre cosine-to-target (0.7) but excellent neighbor overlap (0.9) — it got the relationships right while sitting slightly off the exact points. This is a success for retrieval.
• A map can have high cosine-to-target on average (0.9) but scramble the local ranking of near-ties — good reconstruction, poor preservation of the thing that matters on hard cases.

We usually do not care whether T(x_A) = x_B. We care whether neighbors remain neighbors, rankings remain rankings, and clusters remain clusters. Optimize and evaluate for that.

Demonstration: four maps on RELATE

MEASURED on RELATE v0.1, Wave 3 rows 3.3 and 3.5 — artifacts null-map-baseline.json and nonlinear-vs-linear-unpaired.json under experiments/embeddings-from-first-principles/wave3/artifacts/.

First the baseline that every map must beat — the null map T(x) = x, for two encoders of the same width (BGE-large → mxbai, split_entity:test):

null map T(x) = x    coordinate cos 0.98   10-NN overlap 0.88   retrieval ratio 0.995

Doing nothing already aligns two same-width retrieval encoders almost perfectly — they were pre-aligned by their shared objective. A map only earns its place when it beats this.

Now four maps, all-mpnet-base-v2 → bge-large-en-v1.5 (different family), anchors from split_entity:train, evaluated on held-out entities:

map                    coord cos   10-NN overlap   relation-order corr   calibration transfer   fit
orthogonal Procrustes     0.40          0.73              0.91                  0.98            closed form
ridge (linear)            0.79          0.68              0.83                  0.83            closed form
nonlinear MLP (2×512)     0.74          0.67              0.89                  0.83            ~1 s, seed-variant

MEASURED: the nonlinear MLP does not win. Procrustes keeps the most neighborhood and relation-ordering structure (it only rotates, so angles survive); ridge reconstructs coordinates best but scrambles more near-ties. The MLP lands between them on every metric, costs a training run, and its result moves with the seed (±0.007). Expressiveness bought nothing here that a closed-form linear map did not already have — the “number of parameters that could overfit” grew by four orders of magnitude for no gain. Chapter 21 confirms the pattern holds for the hard-negative margin too.

What this chapter establishes and what it does not

Establishes: the alignment map family (null map ⊂ Procrustes ⊂ linear ⊂ CCA-subspace ⊂ nonlinear), plus “re-express, don’t map” (relative representations) as an alternative to fitting a map; the preprocessing that matters (centering, normalization, anchor coverage, dimension handling); the coordinate-reconstruction vs. semantic-preservation distinction, and that they routinely diverge; that nonlinearity helps unpaired-alignment optimization more than it helps the final map’s capacity (vec2vec vs mini-vec2vec).

Does not establish: which map is best (target- and data-dependent), or that more expressive is better (it usually is not, per generalization risk). It establishes the rule: most constrained map that meets the preservation target, evaluated on preservation not reconstruction.

Lab 19: map bake-off

PROPOSED, not executed.

Setup. Two models, 1,000+ dual-embedded objects, train/test anchor split, hard-negative subset, labeled queries.

Task.

1. Fit Procrustes, linear regression, CCA, and a 2-layer MLP on the same train anchors, same preprocessing.
2. On held-out anchors: cosine-to-target, neighbor overlap@10, rank correlation.
3. Downstream: retrieval agreement@10 vs native E_B, overall and hard negatives.
4. Repeat with all-easy anchors vs. coverage-balanced anchors; note the difference.

Success criterion. The most constrained map whose preservation metrics are within noise of the best, plus a statement of how much anchor coverage (not map complexity) moved the hard-negative number.

Companion component: the bridge — method and preprocessing

bridge (v1):
  ...v0 fields...
  method:            <null | procrustes | linear | cca(k) | relative_reps(anchor_set) |
                      mlp(layers) | unpaired_iterative | vec2vec | mini_vec2vec>
  paired:            <true | false>          # false = unpaired (pseudo-anchor / vec2vec-style)
  null_map_baseline: {cos, top1, retrieval_agreement}   # if this is already good, the spaces were pre-aligned
  preprocessing:     {center: bool, normalize: <...>, source_pca: <matrix_hash|none>}
  anchor_selection:  {strategy, coverage_report}
  reconstruction_metrics:  {cos_to_target, mse}
  preservation_metrics:    {nbr_overlap, rank_corr, cluster_agreement, retrieval, hard_neg}
  chosen_because:     "most constrained map within noise of best preservation"

The Observatory records both metric families separately and displays the preservation family by default — reconstruction is diagnostic, not the goal.

Failure modes

• Skipping the null map. If T(x) = x already scores well, the two spaces were nearly identical and every metric on your fitted map is inflated. Always report the null-map baseline.
• Optimizing MSE to the target vector. That is reconstruction; you may be scrambling the rankings that matter.
• Jumping to an MLP. Fit the constrained maps first; only escalate on a measured plateau. In the unpaired regime nonlinearity may still help the optimization — that is a separate reason from capacity.
• All-easy anchors. The bridge inherits their blind spots.
• Procrustes without scale matching. It assumes comparable scale; normalize first.
• Forgetting the source PCA in the identity. If you PCA-reduced space A before Procrustes, that projection is now part of the bridge and must be recorded.

What this chapter established

• The alignment family: null map, orthogonal Procrustes, linear regression, CCA, nonlinear MLP — ordered by expressiveness and by overfitting risk — plus relative representations (“re-express, don’t map”). Always run the null map first: a good null-map score means the spaces were already nearly identical.
• Preprocessing (centering, normalization, anchor coverage, dimension handling) often matters more than map choice.
• Coordinate reconstruction (cosine/MSE to target) and semantic preservation (neighbors, rankings, clusters, retrieval) are different and routinely diverge.
• Nonlinearity helps the unpaired alignment optimization converge more than it adds capacity to the final map (vec2vec’s nonlinear translator vs mini-vec2vec’s linear one reach comparable coarse quality).
• The rule: most constrained map meeting the preservation target; the bridge records both metric families and foregrounds preservation.

Next

We can fit maps and measure them. The next chapter assembles this into an explicit artifact — the embedding bridge — with source and target hashes, a method, a status, calibration, and a scoped list of what it is usable for, so a bridge never silently claims more compatibility than it has.