RELATE: Searching Embeddings by Relation, Not Just Similarity

Embeddings are everywhere in modern AI.

They power semantic search, retrieval-augmented generation, recommendations, clustering, duplicate detection, code search, memory systems, and many of the mechanisms through which an AI system decides what information is relevant.

Yet most systems interrogate embeddings in essentially the same way:

Take two vectors and calculate cosine similarity.

That is useful. But it also makes a strong assumption.

It assumes that the information we care about is expressed directly through the default geometry of the embedding space.

Our research began with a simple question:

What if an embedding contains useful information that cosine similarity does not expose?

We now have a concrete answer.

On a frozen benchmark of real Python functions, cosine distance correctly ordered the structurally closer candidate 53.25% of the time. Euclidean distance over the same embeddings achieved 53.33%.

A small ridge projection trained to predict three measurable properties of code achieved 73.29% pairwise ordering accuracy—an absolute gain of 19.95 percentage points over the stronger raw-distance baseline.

The encoder did not change. The underlying 768-dimensional embeddings did not change.

The readout did.

There is an important asymmetry in this comparison. Cosine and Euclidean distance received no information about the selected relation. RELATE was trained on AST-derived structural coordinates from the training split.

This is therefore not an apples-to-apples contest between two unsupervised distance metrics. The experiment asks a narrower question:

Can a small supervised readout recover relation-specific information from a frozen embedding that its default geometry exposes poorly?

For this code-structure relation, the answer was yes.

We did not fine-tune the encoder, generate a second embedding representation, use an LLM judge, or expose the correct test ordering during evaluation. We trained a small readout on the training split and then evaluated it against frozen test queries and frozen candidate pairs.

That is the idea behind RELATE.

👉 GitHub repository
👉 Interactive Hugging Face demo

First: what is an embedding?

An embedding is a numerical representation of an item such as a word, paragraph, image, or piece of code.

An embedding model converts that item into a vector containing hundreds or thousands of numbers. In the CodeBERT experiment described here, each Python function was represented by 768 numbers.

You can think of those numbers as coordinates in a very large space.

Items that the model has learned to represent similarly will often occupy nearby regions of that space. That makes embeddings useful for search, recommendations, clustering, duplicate detection, and retrieval-augmented generation.

But the individual coordinates do not normally correspond to neat human concepts such as:

dimension 14 = complexity
dimension 82 = formality
dimension 417 = security risk

The information is distributed across many coordinates and entangled with other information.

To compare two embeddings, most systems calculate cosine similarity. Cosine measures whether two vectors point in broadly the same direction and reduces their relationship to a single score.

That score is useful, but it is not the embedding itself.

The embedding contains hundreds of coordinates. Cosine similarity is one mathematical rule for reading them.

The central question behind RELATE is therefore:

Could the same frozen coordinates support another readout—one designed for the particular relation we care about?

In our code experiment, the selected relation was structural similarity. We asked whether the embedding could predict three measurable properties of a Python function:

• cyclomatic complexity;
• maximum control-flow depth;
• number of distinct call sites.

The embedding remained unchanged.

Only the method used to interpret it changed.

The original mistake

The first version of this research became much larger than it needed to be.

We were investigating whether similarity was relative: whether the correct definition of “close” depended on the relation being measured.

That led naturally into a wide range of questions:

• Could embeddings expose multiple geometries?
• Could we construct primitive relation coordinates?
• Could we learn operators over those coordinates?
• Could we improve semantic search?
• Could this help detect hallucinations?
• Could we discover latent relations automatically?
• Could several models verify one another?
• Could we build an entire experimental framework around the idea?

Some of those questions remain interesting.

They were also distracting us from the smallest result that mattered.

The minimal claim was not:

We have discovered a universal new embedding geometry.

It was not:

Cosine similarity is obsolete.

It was not:

Every useful relation can be extracted from every embedding.

The smallest claim was this:

Given a useful measurable relation, a projection learned over frozen embeddings may expose information that cosine similarity misses.

That claim can be tested directly.

So we started again.

What RELATE actually does

RELATE is deliberately small.

Suppose we have:

1. a set of frozen embeddings;
2. one or more measurable coordinates describing a relation we care about;
3. new embeddings that we want to search.

We fit a projection:

embedding → relation coordinates

How RELATE Reads the Same Embedding Differently

    flowchart TD
    %% ── Training ──
    subgraph TRAINING["🧪 1. Learn the relation‑specific readout"]
        TrainCode["📄 20,000 Python functions"]
        TrainEmbedding["🧠 Frozen CodeBERT embeddings<br/>768 dimensions"]
        TrainAST["📏 Measured AST coordinates<br/>complexity · control depth · call sites"]
        Ridge["📊 Fit three Ridge projections"]

        TrainCode -->|"🧊 Frozen CodeBERT"| TrainEmbedding
        TrainCode -->|"🌳 Parse code — never execute it"| TrainAST
        TrainEmbedding --> Ridge
        TrainAST -->|"🎯 Training targets"| Ridge
    end

    Ridge -.->|"🔒 Freeze the learned coefficients"| Projection

    %% ── Search ──
    subgraph SEARCH["🔎 2. Search the same frozen embeddings in two different ways"]
        Query["🔍 Query function"]
        Candidates["🎯 Candidate functions"]

        QueryEmbedding["🧠 Query embedding"]
        CandidateEmbeddings["🧠 Candidate embeddings"]

        Cosine["📐 Cosine distance<br/>default embedding geometry"]
        Projection["🌌 RELATE projection"]
        Predicted["📈 Predicted relation coordinates"]
        RelationDistance["📏 Chebyshev distance<br/>in relation space"]

        Query -->|"🧊 Frozen CodeBERT"| QueryEmbedding
        Candidates -->|"🧊 Frozen CodeBERT"| CandidateEmbeddings

        QueryEmbedding --> Cosine
        CandidateEmbeddings --> Cosine

        QueryEmbedding --> Projection
        CandidateEmbeddings --> Projection
        Projection --> Predicted
        Predicted --> RelationDistance
    end

    %% ── Evaluation ──
    subgraph EVALUATION["📊 3. Evaluate the rankings"]
        CosineRanking["🏆 Cosine ranking"]
        RelateRanking["✨ RELATE ranking"]
        TrueCoordinates["✅ True AST coordinates"]
        CorrectOrdering["🧪 Which candidate is structurally closer?"]

        Cosine --> CosineRanking
        RelationDistance --> RelateRanking
        Candidates -->|"🌳 Parse AST"| TrueCoordinates
        TrueCoordinates --> CorrectOrdering

        CosineRanking --> Compare["⚖️ Compare against the frozen correct ordering"]
        RelateRanking --> Compare
        CorrectOrdering --> Compare
    end

    %% ── Styling classes (names only – define styles in CSS) ──
    class TrainEmbedding,QueryEmbedding,CandidateEmbeddings brainNode
    class Cosine,RelationDistance distNode
    class Projection,RelateRanking highlightNode
    class CosineRanking,Compare compareNode
  

RELATE does not alter the embedding model. It learns a small readout from frozen embeddings into measurable relation coordinates. At evaluation time, cosine similarity and RELATE rank the same candidates using the same underlying embeddings; only the method used to interpret those embeddings changes.

We then search in the predicted relation space rather than relying entirely on the original embedding geometry.

For the real-code result, the relation coordinates were three objective properties of Python functions:

1. cyclomatic complexity;
2. maximum control-flow nesting depth;
3. number of distinct call sites.

These values can be extracted from the Python abstract syntax tree. They do not require human preference labels or an LLM evaluator.

The training process was therefore:

frozen CodeBERT embedding
        ↓
ridge projection
        ↓
predicted structural coordinates

At search time, a query and candidate were both projected into those coordinates.

Their distance was calculated using Chebyshev distance: the largest mismatch across the three robust-scaled coordinates.

In simplified form:

distance(query, candidate)
    =
max(
    complexity difference,
    depth difference,
    call-site difference
)

This asks a different question from cosine similarity.

Cosine asks:

Are these vectors pointing in a similar overall direction?

RELATE asks:

According to the relation we selected, how far apart are these two items?

Those are not equivalent questions.

The real-code result

The benchmark used frozen CodeBERT embeddings for Python functions.

The canonical data contained:

20,000 training functions
4,000 validation functions
4,000 test functions
768 embedding dimensions

The evaluation contained:

4,000 test queries
512,000 frozen hard-negative comparisons
128 comparisons per query

The task was pairwise structural ordering. Given a query function and two candidates, each method had to identify which candidate was closer to the query under the selected code-structure relation.

How the hard negatives were constructed

The hard negatives were constructed before method evaluation and without consulting the embedding geometry.

For each test query:

1. training candidates were restricted to the same token-length decile as the query;
2. candidates were ordered by Chebyshev distance over the robust-scaled true structural coordinates;
3. pairs with equal true distance were excluded;
4. eligible pairs had to be separated by between 5 and 25 positions in the true-coordinate ranking;
5. up to 128 eligible pairs were selected using a deterministic SHA-256 ordering.

The construction process did not load CodeBERT embeddings, predicted structural coordinates, cosine distances, Euclidean distances, token-length results, or any test-performance result.

The benchmark was therefore not created by searching for examples on which cosine failed. Its difficulty came from distinguishing candidates that were relatively close in an independently frozen structural ranking, while controlling broadly for token length.

For each comparison, a method received:

1.0  when it ranked the structurally closer candidate first
0.5  when its two candidate distances tied
0.0  when it ranked the farther candidate first

Scores were averaged within each query and then averaged equally across all 4,000 test queries. Chance-level pairwise ordering accuracy is 0.500.

How structural closeness was defined

The correct ordering was defined using Chebyshev distance over three robust-scaled true coordinates:

• cyclomatic complexity;
• maximum control-flow nesting depth;
• distinct call sites.

RELATE used the same distance family over its predicted versions of those coordinates. This matching was intentional: the experiment tested whether the frozen embedding could predict enough of the selected relation to reproduce its ordering.

The true-coordinate method achieves 1.0 by construction. It is included as a ceiling showing the remaining gap between predicted and perfectly known relation coordinates, not as independent validation.

RELATE received structural supervision during training. Cosine and Euclidean distance did not. The comparison therefore tests whether a small supervised readout can recover information from the frozen representation that its raw geometry exposes poorly.

Results

RELATE improved pairwise ordering accuracy from approximately 53.33% for the strongest raw embedding-space baseline to 73.29%—an absolute gain of 19.95 percentage points.

The predicted coordinates were not perfect. A substantial gap remains between RELATE’s score of approximately 0.733 and the true-coordinate ceiling of 1.0.

But the result establishes the narrower claim under test: the frozen CodeBERT embeddings contained recoverable information about this code-structure relation that raw cosine and Euclidean distance exposed poorly.

Try RELATE yourself

The benchmark result is the scientific evidence, but the underlying mechanism is easier to understand when you can see it operate.

We have published an interactive Hugging Face demo:

👉 Open the RELATE Hugging Face demo

The demo accepts:

1. a query Python function;
2. Candidate A;
3. Candidate B.

It then embeds all three functions using the same frozen CodeBERT model and compares the candidates in three different ways:

• Cosine distance — the default geometry of the original embeddings;
• RELATE distance — distance after projecting those embeddings into predicted structural coordinates;
• True AST distance — distance calculated from the measurable structure of the submitted functions.

The predicted coordinates are:

• cyclomatic complexity;
• maximum control-flow depth;
• number of distinct call sites.

The demo therefore lets you see cases where cosine and RELATE agree, and cases where they read the same embeddings differently.

It also shows the predicted structural coordinates beside the values measured directly from the Python abstract syntax tree.

Submitted code is parsed and embedded, but it is never executed.

What the demo does not prove

The live examples are illustrations, not a replacement for the frozen benchmark.

A function pasted into the demo may differ considerably from the data on which the projection was trained. A successful individual example does not establish a new experimental result, and an unsuccessful example does not invalidate the preserved benchmark.

The evidence-bearing result remains the frozen evaluation over:

4,000 test queries
512,000 hard-negative comparisons

The purpose of the demo is narrower:

To make visible how the same frozen embedding can produce one ranking under cosine similarity and another under a relation-specific readout.

Why this matters

An embedding is not a single score.

It is a high-dimensional representation.

When we calculate cosine similarity, we compress the interaction between two embeddings into one scalar according to one fixed rule.

That scalar may be useful while still discarding information.

Consider a crude analogy.

Imagine a database describing houses with hundreds of attributes:

• location;
• floor area;
• number of rooms;
• energy efficiency;
• age;
• garden size;
• distance from schools;
• construction type.

A general similarity score might identify houses that are broadly alike.

But a buyer searching specifically for houses with similar heating requirements is asking a narrower question.

The information may already be present in the database. The general similarity function may simply fail to prioritise it.

Embeddings may behave similarly.

Their coordinates can encode many overlapping properties. The default geometry combines those properties into a general notion of proximity.

A relation-specific projection attempts to recover one selected property and search according to it.

This does not make cosine similarity wrong.

For relation-specific retrieval, cosine similarity can be incomplete.

Related earlier evidence: alternative readouts do not always help

These were not direct evaluations of the RELATE coordinate-projection method.

Before the real-code structural experiment, we had tested the broader premise on two pair-classification benchmarks using a different embedding model, different metrics, and different pair-feature mechanisms.

They are included as related historical evidence, not as independent confirmation of the RELATE result.

On PAWS, the alternative embedding-pair representation performed worse than cosine. On BigCloneBench, embedding-pair features substantially outperformed cosine.

Together, those earlier experiments helped motivate the narrower question tested here:

Can a known, measurable relation be recovered from frozen embeddings even when the default geometry exposes it poorly?

They also warn against a universal claim. Alternative readouts do not automatically help. Their value depends on the representation, relation, supervision, task, and evaluation design.

The first was PAWS, a paraphrase benchmark.

The frozen result was:

The larger pair representation performed worse than cosine:

-0.0260

The second was a BigCloneBench code-clone benchmark.

The frozen result was:

Here, the embedding-pair features substantially outperformed cosine.

The full-pair gain was:

0.1305

These results are important together.

They tell us that “there is always more useful information than cosine reveals” would be too strong.

On PAWS, the tested additional representation did not help.

On BigCloneBench, it helped considerably.

On the real-code structural benchmark, a supervised projection into explicit relation coordinates produced the strongest observed gain.

The evidence therefore supports a conditional claim:

Some relations are encoded in an embedding but poorly exposed by its default similarity geometry.

That is narrower than a universal theory.

It is also far more useful.

Reproducing the evidence

One danger in AI research is that the headline result survives while the exact path that produced it disappears.

Models change.

Datasets move.

Local embedding files are deleted.

Random seeds are forgotten.

Preprocessing code drifts.

A result slowly becomes a story that nobody can reproduce.

We nearly allowed that to happen.

The original work had accumulated across multiple repositories, experiment frameworks, local caches, canonical manifests, generated predictions, and frozen evaluation artifacts.

Instead of starting another experiment, we recovered the original evidence.

PAWS and BigCloneBench

The preserved embedding snapshots and manifests were replayed in a new minimal implementation.

The replay reproduced the historical values exactly, including:

PAWS:
cosine       = 0.6205
full pair    = 0.5945
difference   = -0.026000000000000023

BigCloneBench:
cosine       = 0.7255
full pair    = 0.856
difference   = 0.13049999999999995

The negative result reproduced exactly.

The positive result reproduced exactly.

That distinction matters. A replay mechanism that reproduced only the favourable result would be much less convincing.

The historical real-code benchmark

The original CodeBERT embeddings were recovered from two independently generated SQLite caches.

For each cache, the replay verified every frozen row’s:

• stable key;
• source-code hash;
• extraction fingerprint;
• dtype;
• embedding dimension;
• binary payload hash;
• vector-array hash.

Both caches independently reconstructed the exact canonical matrices.

The reconstructed matrix hashes matched the frozen canonical hashes.

The replay then reloaded:

• the selected row manifests;
• the primitive-coordinate tables;
• the predicted training coordinates;
• the predicted test-query coordinates;
• the 4,000 hard-negative query records;
• the 512,000 frozen candidate pairs;
• the previously published independent verification.

It recomputed the complete primary score array from scratch.

The replayed decision exactly matched the independently published historical decision.

The final status was:

OPTION_B_HISTORICAL_REPLAY_COMPLETE

No embedding was regenerated.

No model was downloaded.

No probe was refitted.

No query, pair, method, threshold, or decision rule was changed.

A necessary boundary

One later experiment in the broader research programme, RELATE-E01, was not successfully executed.

Its terminal status remains:

EXPERIMENT_INVALID

The primary test was never opened.

The results described here do not repair, reinterpret, or replace E01.

They belong to separate preserved evidence:

• external pair-benchmark replays;
• the historical Option B real-code result.

This boundary is not administrative trivia.

A failed experiment should remain failed.

The correct response is not to blur it into a successful neighbouring result. The correct response is to preserve what actually happened and build from evidence that remains valid.

From research project to implementation

The new RELATE repository does not contain the old experimental machinery.

The implementation is intentionally small.

At its centre is a relation projection:

model = RelationProjection.fit(
    training_embeddings,
    training_relation_coordinates,
    relation_names=(
        "cyclomatic_complexity",
        "max_control_depth",
        "distinct_call_sites",
    ),
)

Once fitted, it can search target embeddings in the learned relation space:

hits = model.search(
    source_embedding,
    target_embeddings,
    k=10,
)

The implementation also supports first-stage candidate filtering followed by relation-specific reranking. That is an engineering capability, not a result established by this benchmark. Before using cosine as the first-stage filter for structural retrieval, candidate-pool recall should be measured against the true structural neighbours.

The deeper idea

Modern AI systems often treat embeddings as though their meaning were exhausted by nearest-neighbour search.

One useful way to think about an embedding is as a compressed evidence field.

Different questions may require different readout functions.

Cosine similarity is one readout.

A linear classifier is another.

An elementwise interaction is another.

A projection into measurable coordinates is another.

The important question is no longer simply:

Which items are most similar?

It becomes:

Similar according to which relation?

Two code functions may be:

• semantically similar but structurally different;
• structurally similar but semantically unrelated;
• similar in complexity but different in call behaviour;
• similar according to one engineering risk but not another.

A single global similarity score cannot faithfully represent every one of those relationships at once.

RELATE does not solve that entire problem.

It demonstrates one concrete case in which this problem appears.

What this evidence supports

We should be precise.

We have not proved that every frozen embedding contains all relations of interest.

We have not proved that linear projection is always sufficient.

We have not proved that structural relation search will improve every downstream application.

We have not proved that cosine should be removed from retrieval systems.

We have shown that:

1. frozen embeddings can contain recoverable information that cosine similarity exposes poorly;
2. this effect can be substantial on a real-code relation benchmark;
3. the useful relation can be expressed through objective, independently measurable coordinates;
4. a small ridge projection can recover enough of those coordinates to materially improve hard-negative ordering;
5. the complete historical result can be reproduced exactly from preserved inputs;
6. the effect is relation- and benchmark-dependent rather than universal.

The central result is therefore not that cosine similarity failed.

It is that the embedding knew more than its default geometry showed.

What stronger evidence still requires

This experiment establishes recoverability from the frozen embedding. It does not establish that the embedding is the cheapest source of the signal, that this projection is the strongest supervised method, or that the result generalises across corpora and encoders.

The most important next controls are:

1. a lexical or conventional code-feature baseline;
2. a supervised pairwise or metric-learning baseline over the same frozen embeddings;
3. query-level bootstrap confidence intervals;
4. project-level or near-duplicate-resistant data splits;
5. ablations over individual coordinates and relation-space distance functions;
6. candidate-pool recall measurements for any proposed two-stage retrieval architecture.

These controls may strengthen, narrow, or overturn the practical interpretation. They do not need to be assumed in advance. They need to be run and published.

Where RELATE goes next

The next stage should not be another sprawling research programme.

It should be a sequence of small, publishable demonstrations.

For each relation:

1. define the relation clearly;
2. identify objective or defensible coordinates;
3. freeze the embeddings;
4. compare against cosine and Euclidean distance;
5. evaluate on hard cases;
6. publish the result whether it succeeds or fails.

Possible relation families include:

• code complexity;
• control-flow shape;
• API usage;
• security-sensitive behaviour;
• document formality;
• evidential support;
• temporal proximity;
• causal role;
• contradiction;
• writing style;
• operational risk.

Some will fail.

That is part of the point.

RELATE is not a claim that every relation is latent and recoverable.

It is a mechanism for testing which relations are.

Conclusion

AI systems rely heavily on embeddings, but often interrogate them through one default geometry.

Our result shows that this can leave useful evidence hidden in plain sight.

On a frozen CodeBERT benchmark, raw cosine and Euclidean distance performed only slightly above chance on structural hard negatives.

A small projection into three predicted code-structure coordinates improved ordering accuracy from approximately 0.533 to 0.733.

RELATE does not show that cosine similarity is obsolete, that every useful relation is recoverable, or that a ridge projection is always the best readout.

It shows one concrete case in which a frozen embedding contained substantially more relation-specific information than its default geometry revealed.

On structural hard negatives, cosine and Euclidean distance achieved approximately 53.3% pairwise ordering accuracy. A small supervised readout over the same frozen embedding coordinates achieved 73.3%.

The encoder did not change.

The embeddings did not change.

The question—and the readout used to answer it—did.

That is the practical idea behind RELATE:

Do not ask only which items are generally similar. Identify the relation that matters, determine whether the embedding contains evidence for it, and test a readout designed to expose that evidence.