Research Coda — What Alignment Really Preserves

“Preserve the geometry” is not a complete instruction

A bridge translates vectors from space A into space B.

The natural instinct is to say:

Preserve the source geometry while you translate.

That sounds obviously correct. If two source points are close, keep them close. If two source points are far apart, keep them far apart. Preserve pairwise cosine, distances, neighborhoods.

But Chapter 16 established something uncomfortable: different encoders do not merely rotate the same universe. They can disagree on neighborhoods, density, rank order, hard distinctions, and calibration.

If A and B genuinely organize the same objects differently, preserving A’s geometry can prevent the translated vectors from behaving like B.

So the real question is not:

How do we preserve the geometry?

It is:

Which geometry should the translation preserve, and which downstream behavior is the preservation claim supposed to protect?

Three spaces are involved, not two

For paired objects x_i, define:

A_i = E_A(x_i)          source-native vector
B_i = E_B(x_i)          target-native vector
T_i = T(A_i)            translated source vector, living in B coordinates

Now there are three relational structures:

G_A = relationships among A_i
G_B = relationships among B_i
G_T = relationships among T_i

A translation objective can ask for different equalities:

source-isometry:       G_T ≈ G_A
point reconstruction: T_i ≈ B_i
 target fidelity:      G_T ≈ G_B
 task fidelity:        decisions(T_i) ≈ decisions(B_i)

These objectives can agree.

They can also fight each other.

A tiny conflict

Suppose source encoder A sees:

A-space neighborhood of x

1. paraphrase
2. same topic
3. negation

Target encoder B sees:

B-space neighborhood of x

1. paraphrase
2. entailment
3. same topic

The negation is no longer in the top three.

Now impose a loss saying translated A must preserve A’s pairwise cosine relationships exactly.

That loss insists the negation remain close.

But reproducing B’s geometry requires moving it away.

The bridge cannot satisfy both perfectly because the encoders disagree about the local structure.

This is not an optimization bug. It is a conflict between invariants.

Four preservation objectives

1. Preserve the source shape

A source-geometry loss compares pairwise relationships before and after translation:

S_A(i,j) = cos(A_i, A_j)
S_T(i,j) = cos(T_i, T_j)

L_source = mean((S_A - S_T)^2)

This says: translation may move the cloud, but must not change its internal shape.

That is exactly right when A and B are believed to be the same geometry in different coordinates — for example, an orthogonal basis change.

It is a bad assumption when the encoders’ native neighborhoods differ materially.

2. Reconstruct the target points

With paired data:

L_point = distance(T_i, B_i)

This says: put each translated item where B put the same item.

It directly supports counterpart recovery. It does not guarantee that near-ties, neighborhoods, or clusters are right, as Chapter 25 showed.

3. Reproduce the target geometry

Instead of preserving G_A, use G_B as the teacher:

S_B(i,j) = cos(B_i, B_j)
S_TB(i,j) = cos(T_i, B_j)

Then optimize the translated point to induce the same ranking over target items as the native target point.

One differentiable version is distributional rank distillation:

p_j = softmax(S_B(i,j)  / τ)
q_j = softmax(S_TB(i,j) / τ)

L_target_rank = KL(p || q)

This does not say the source geometry is sacred. It says the target encoder’s neighborhood behavior is the thing to imitate.

That is a better match when the consumer will search a B-space index.

4. Preserve the downstream task

Sometimes neither geometry is the real target.

For retrieval:

L_task = ranking loss against relevance labels

For clustering:

L_task = cluster-consistency objective

For a calibrated threshold:

L_task = operating-point / FAR / FRR preservation

This says: preserve what the consumer actually uses, even if both native geometries are imperfect proxies.

Demonstration: the “geometry-preserving” loss made the bridge worse

MEASURED on the frozen 5,264-sentence cross-space benchmark, HNet→Qwen, deterministic 4,199/1,065 split. A paired neural oracle is used here only as a diagnostic: pairing tells the model the correct correspondence so the experiment isolates the effect of the preservation loss rather than the difficulty of unpaired discovery.

First remove a capacity confound. A 256-dimensional latent bottleneck could not match the paired linear baseline. Increasing the latent width to 1,024 allowed the paired neural map to reach almost the same pointwise cosine as linear regression:

                             agree@10   Recall@10   cosine   order
paired linear                   0.589      0.997      0.819    0.64
paired neural, 1024 latent      0.567      0.966      0.816    0.61

The neural map is not better, but it is close enough to serve as a controlled platform for asking what a loss does.

Now add cosine-based vector-space preservation — force the translated cloud to keep source-space pairwise cosine relationships:

                             agree@10   Recall@10   cosine   order
paired neural, point loss       0.567      0.966      0.816    0.61
+ source cosine preservation    0.177      0.100      0.460    0.43

The result is not a small regression.

Neighborhood agreement falls by roughly 39 points. Paired-target Recall@10 falls from 0.966 to 0.100. Cosine-to-target collapses. Order preservation degrades sharply.

On this benchmark, preserving source cosine geometry actively prevents the map from approximating the target representation.

A preservation loss can be perfectly faithful to the wrong invariant.

Why this result matters more than the particular neural model

The interesting conclusion is not “this MLP failed.” The paired model without the preservation term already demonstrates that the architecture can approximate the pointwise map reasonably well.

The diagnostic intervention changes one idea:

without VSP:  move source points toward target points
with VSP:     move source points toward target points
              WHILE preserving source pairwise cosine

Performance collapses when the second requirement is added.

That supports a narrow but important conclusion:

On this source/target pair, source-space cosine relationships are not the right invariant to force onto the target space.

It does not prove every source-geometry loss is harmful. It does show that “preserve pairwise cosine” cannot be treated as an automatically beneficial regularizer.

The unpaired problem is a different problem

There is another distinction worth keeping clean.

A paired bridge is told which object in A corresponds to which object in B. An unpaired bridge sees two point clouds and must discover the correspondence as well as learn the map.

On the same frozen benchmark, several unpaired objectives — reconstruction, cycle consistency, adversarial distribution matching, and combinations with cosine preservation — stayed near chance on neighborhood agreement:

best unpaired agreement@10    0.067
best paired linear            0.589
advancement target            0.670

This does not establish that unpaired translation is impossible. It establishes something more useful for engineering:

At this data scale and for this model pair, the unpaired objectives did not identify the correspondence that paired regression could exploit immediately.

That separates two questions:

Can this map family REPRESENT a useful transport?
Can this training signal DISCOVER the transport without pairs?

Do not diagnose the second failure as the first.

When source preservation is the right choice

Source geometry should be preserved when source relationships are themselves the contract.

Examples:

• Coordinate migration inside one known geometry. You are changing basis, precision, or storage format and explicitly want an isometry.
• Round-trip archival transport. A→B→A should recover the original relationships.
• A source-calibrated consumer remains authoritative. The target is merely a computational carrier, not a new semantic standard.
• The two native spaces have already been shown to have high structural agreement. If G_A ≈ G_B, preserving either is mostly consistent.

In those cases an orthogonal map or source-VSP objective has a clear semantic justification.

When target preservation is the right choice

Target geometry should be the teacher when the translated vectors will be consumed as if they were native target vectors.

Examples:

• searching an existing B-space index;
• migrating legacy vectors to a new encoder generation;
• feeding translated vectors into a B-trained clustering or routing system;
• replacing native B embeddings in a downstream component whose behavior is already calibrated against B.

Then the bridge’s obligation is not “do not disturb A.” It is:

behave like B where the consumer looks

That may require deliberately changing A’s neighborhood structure.

When task preservation should override both

Suppose A and B both place a negated claim too close to its assertion. Faithfully reproducing either geometry preserves the same mistake.

If the application actually cares about polarity, the correct target is neither G_A nor G_B. It is the task relation:

relevant evidence above contradictory evidence

A supervised task-aware bridge can reweight information already present in the source representation and outperform the source encoder’s naive cosine decision (Chapter 21’s supervised hard-negative result).

So the hierarchy is:

consumer behavior, when labels exist
        ↑
target geometry, when native target behavior is the contract
        ↑
source geometry, when source relationships are the invariant

There is no universally correct preservation loss outside that context.

Choose the invariant before choosing the loss

A bridge specification should therefore contain not just a method but a declared preservation objective:

preservation_objective:
  authority: <source_geometry | target_geometry | downstream_task>
  property:  <pairwise_cosine | neighborhood_rank | cluster | ndcg | calibration | ...>
  reference_space_hash: <hash or none>
  reference_dataset_hash: <hash>
  evaluation_metric: <metric>
  success_bar: <value>

This prevents a common category error: optimizing an elegant geometric quantity that the downstream consumer never actually uses.

A target-rank objective

The next obvious experiment is not “a bigger GAN.” It is to optimize the failure metric directly.

For each paired training object i:

1. Compute native target similarities from B_i to a candidate set {B_j}.
2. Compute translated similarities from T(A_i) to the same {B_j}.
3. Distill the target ranking or target similarity distribution.
4. Keep a pointwise term so the paired target remains identifiable.

Conceptually:

L = λ_point L_point
  + λ_rank  L_target_rank
  + λ_task  L_task       # optional

Crucially, there is no term saying G_T must equal G_A unless source geometry is explicitly part of the consumer contract.

This is a proposal, not a measured result. The measured result is the diagnosis that motivates it.

Failure modes

• Treating “preserve geometry” as self-explanatory. Name source, target, or task geometry.
• Using source pairwise-cosine preservation when source and target neighborhoods disagree. You may be regularizing away the target behavior.
• Selecting checkpoints by cosine when the consumer reads rankings. Optimize and select by agreement/order/nDCG if those are the contract.
• Calling an unpaired discovery failure a capacity failure. A paired oracle distinguishes representability from identifiability.
• Adding more adversarial machinery before checking the invariant. A stronger optimizer cannot rescue the wrong objective.

What this chapter establishes and what it does not

Establishes: source-isometry, point reconstruction, target-geometry fidelity, and task fidelity are different objectives; they can conflict when encoders organize the same objects differently; on the measured HNet→Qwen benchmark, adding source cosine-VSP to an otherwise useful paired neural map catastrophically reduced target recovery and neighborhood agreement; a preservation objective should name its authority and consumer.

Does not establish: that source-geometry preservation is always harmful; that target-rank distillation will solve the problem (it is the next proposed experiment, not a result); that unpaired translation cannot work at larger scale or with other objectives; or that target geometry is ground truth. It establishes how to choose what a translator owes its destination.

Lab 26: make two preservation objectives disagree

PROPOSED, not executed.

Setup. Two paired embedding spaces over the same 1,000+ objects. First measure native A-vs-B agreement@10; choose a pair where it is materially below 1.0.

Task. Fit the same-capacity paired translator under three losses:

A. point reconstruction only
B. point reconstruction + SOURCE cosine preservation
C. point reconstruction + TARGET rank distillation

Evaluate all three on the same held-out set:

Success criterion. Not that C wins. The success criterion is a causal answer: does forcing source geometry help or hurt target behavior, and does a target-defined ranking objective move the metric it directly represents?

Companion component: the preservation objective record

bridge_training_record:
  source_space_hash: <...>
  target_space_hash: <...>
  supervision: <paired | unpaired>

  preservation_objective:
    authority: <source_geometry | target_geometry | downstream_task>
    property: <pairwise_cosine | neighborhood_rank | ndcg | calibration | ...>
    reference_hash: <space or dataset hash>
    loss: <exact formula / implementation version>

  evaluation:
    counterpart_recovery: {...}
    structural_fidelity: {...}
    task_fidelity: {...}

  selected_by: <metric matching the consumer>

The bridge registry can now answer not only “what did this bridge preserve?” but “what was it trained to preserve, and was that the right thing for this consumer?”

What this chapter established

• “Preserve the geometry” is incomplete without naming whose geometry and which property.
• Source-isometry, point reconstruction, target-neighborhood fidelity, and downstream task fidelity are distinct and can conflict.
• On the measured HNet→Qwen benchmark, source cosine-VSP was not merely weak: it collapsed an otherwise useful paired map, demonstrating that a preservation loss can faithfully enforce the wrong invariant.
• A paired oracle separates representational capacity from unpaired identifiability; the unpaired objectives failed to discover a correspondence that paired regression could exploit.
• The right preservation authority comes from the consumer: source geometry for an isometric migration, target geometry for native-target substitution, task labels when the downstream relation is the real contract.
• The next constructive direction is target-rank distillation, proposed rather than claimed.

Closing the coda

Chapter 25 split point recovery from structural fidelity. This chapter split “geometry preservation” into source, target, and task authorities.

Together they sharpen the bridge discipline from Part VI:

A translation does not succeed because it finds the right point, and it does not succeed because it preserves some geometry. It succeeds when it preserves the property the destination consumer actually relies on — measured against the right reference.

That is the same principle the rest of the book keeps arriving at from different directions: geometry is evidence about a representation, not permission to use it.