← Retrieval From First Principles

Understanding Queries and Routing

Choose a bounded retrieval route while preserving the original request and separating intent guesses from evidence outcomes.

“How can people leave a building when an alarm rings?” finds no useful lexical evidence in our new fictional handbook. The relevant sentence is phrased differently: “Emergency evacuation: occupants exit via stairwell; assemble outdoors.” The request and the source describe the same need without sharing the decisive vocabulary. The pinned dense retriever finds the evidence, while an always-lexical route misses it. Choosing a retrieval mechanism can therefore determine whether the required evidence enters the candidate pool at all.

Now consider ZX-68 service reset. Here the literal identifier is part of the need: treating it as expendable could retrieve an instruction for the wrong device. “Which contact serves project PR-84?” presents a third problem. Its answer depends on a supported relationship across records, not merely a sentence that resembles the question. One route need not handle all three demands equally well.

Diversity, Coverage, and Evidence Budgets ended with a finite packet of selected evidence. That packet cannot contain evidence its earlier stages never found. Before ranking and selection, we can therefore ask a distinct question: given this request and the retrieval capabilities available, which mechanism should we invoke under a declared budget?

Interpretation is a proposal

A query expresses an information need, but its wording is an imperfect description of that need. It may contain an exact identifier, suggest an entity or operation, or leave several interpretations open. A router may use those clues to propose a mechanism. It must not confuse its interpretation with permission to change what the user asked.

The plan therefore preserves the original query string and query identity beside a separately recorded interpretation. It retains recognised literal identifiers, inferred entity matches, the chosen route and its reason, uncertainty, allowed sources and work budgets. If later processing produces subqueries, each is recorded as an additional search expression, not a replacement for the original need. That separation makes both the route and any reinterpretation inspectable.

The interface in examples/query_retrieval.py begins with:

plan = retriever.plan(original_query, query_id, allowed_sources)
result = retriever.execute(plan, "policy")
public = retriever.public(result, current_view)

The trusted caller supplies the source allowance. An explicit source constraint in the request may narrow that allowance, but neither query text nor a later expansion may widen it to include protected material. Execution uses a declared view; final projection checks source references and current policy again. The plan describes the route that was authorised to run, not permanent authorisation to disclose everything it finds.

Start with rules whose mistakes are visible

The first router uses an ordered set of visible rules rather than a trained classifier. An ambiguous acronym or entity alias that does not identify one entity leaves the request unresolved. An explicit contact-relationship request chooses structured retrieval; otherwise a device-style literal identifier chooses lexical search. A query beginning “How can” chooses dense search as a paraphrase heuristic, a checksum cue chooses lexical search, and remaining requests use hybrid search. This order is part of the implemented policy, not an incidental detail.

Precedence prevents one useful clue from hiding a more important obligation. A project identifier appears inside a contact request, but the requested contact relationship still requires the structured route. Similarly, “Rowan” is only a display name shared by two registered entities, not an exact entity identifier. Selecting one by a lexical tie-break would disguise unresolved identity as a confident lookup; the plan records the ambiguity instead.

The reasons recorded by these rules explain decisions; they are not probabilities of being right. In particular, “How can” is a fragile wording heuristic, not evidence that dense retrieval will satisfy the need. We have not trained or evaluated an LLM-based intent classifier. The experiment tests what these specific rules retrieve, not whether the system understands user intent in any general sense.

The rules are evaluated in this order and the first match wins. Precedence, not the individual cues, is what keeps an identifier inside a relationship request from overriding the relationship obligation.

    %%{init: {"flowchart": {"wrappingWidth": 470}}}%%
flowchart TD
    R1["1 · ambiguous acronym or entity alias → unresolved, with the ambiguity recorded"] --> R2["2 · explicit contact relationship → structured typed lookups"]
    R2 --> R3["3 · literal device-style identifier → lexical search"]
    R3 --> R4["4 · paraphrase cue 'How can' → dense search"]
    R4 --> R5["5 · checksum cue → lexical search"]
    R5 --> R6["6 · otherwise → hybrid: two searches of three, then RRF"]

Rule four is the heuristic the chapter warns about: it is a wording pattern, and the chapter reports no evidence that it identifies paraphrase needs in general.

Arguello and colleagues’ work on vertical selection distinguishes deciding which resources to search from ranking results inside those resources. That is a useful conceptual separation here, but our experiment makes a different choice: it routes among lexical, dense, hybrid and structured mechanisms over a small declared collection. The cited classifier and its performance are not part of this implementation.

Give the routes comparable limits

All routes operate under the same declared ceilings: at most two logical retrieval calls, six returned generator rows, four final evidence units and 900 selected characters. The text allowance includes two LF characters between selected units. Lexical or dense retrieval can request six candidates in one call; hybrid makes two searches of three and fuses their exact references using the existing RRF rule. A structured route can make two typed adjacency lookups. These limits bound the experiment, although a graph edge and a ranked passage row are not interchangeable units of work.

A shared logical-call ceiling is not a shared runtime or resource cost. Dense encoding, index preparation, graph validation and final policy checks have different work profiles, and the call counter does not include all of them. The trace records actual returned rows, selected text and execution scope. Fewer calls in this table must not be read as a measured end-to-end latency or throughput improvement.

The character allowance was chosen from development unit lengths before the final comparison. It can hold several ordinary retrieval units, deliberately avoiding the earlier whole-document experiment in which the budget rarely admitted more than one item. It does not remove the budget problem: unusually long parent text remains a designed stress case, and selected characters are not a measured model-token cost.

Create opportunities before comparing outcomes

The new evaluation is a constructed mirrored replication, not a return to SciFact and not an independent generalisation test. Twelve development requests and twenty final requests occupy fictional worlds with different literal source, version and anchor identities. The same implementing author created both worlds and could see their labels. Freezing the final files before execution protects the recorded protocol against later adjustment; it does not make the underlying task designs independent.

The overlap between worlds is substantial. Six complete version bodies and six unique query strings recur verbatim. After substituting the explicitly renamed device and main project, seventeen of twenty final query instances match development wording, and nine of eighteen final version bodies match development bodies. Three questions remain unmatched under this narrower map: an unsupported project, an ambiguous organisation name and a misleading project mention. Counting identical wording does not assert identical permissions, revisions or relevance labels.

A second, complete identity-preserving map accounts for the reversed device identifier, project aliases, desk name and each canonical entity ID. Under that map, all eighteen final version bodies correspond to development bodies. The mapping deliberately keeps northern and southern organisations, and main, unsupported and shortcut projects, distinct; the retained calculation lists every correspondence. Renamed identities are not unfamiliar evidence. These results show how the implemented routing mechanisms behave under constructed variants, not how they generalise to independently authored content or needs.

The final requests exercise exact identifiers, terminology, paraphrases, multi-passage obligations, hierarchy and both valid and defective relationships. Sixteen requests declare non-empty target bundles. For three of those targets, removal or restriction leaves required evidence unavailable in the current view; no ranking can assemble a complete eligible bundle for them. Four other requests have no accepted bundle because of ambiguity or unsupported-relationship conditions. Keeping these categories distinct matters when reading a coverage score.

The lexical baseline ranks a unit containing a complete required anchor first for four of the sixteen target needs, and it covers four complete bundles. Thirteen targets have their required evidence structurally available in current block inventories, leaving genuine opportunities to improve retrieval alongside three impossible current-view targets. These are source-anchor and bundle measures, not Part III’s externally annotated document metrics. Their values should not be presented as a cross-part quality comparison.

Separate route labels from retrieval success

The frozen final comparison gives:

Policy Complete target bundles Mean selected characters Mean logical calls
Always lexical 4/16 141.85 1.0
Always dense 7/16 167.75 1.0
Always hybrid 7/16 130.65 2.0
Query-dependent rules 10/16 88.80 1.2

The selected-character and logical-call means include all twenty requests, including those that abstain. Compared with the four lexical successes, the rule policy’s ten include three additional vocabulary-mismatch bundles and three valid relationship bundles. Compared with always-dense retrieval’s seven successes, the policy’s three additional successes all come from invoking structured retrieval for the two main contact requests and the revised contact request, using the contact cue. The result supports adding that capability; it does not by itself support a performance gain from choosing among lexical, dense and hybrid text search.

Remove those three successful structured-relationship requests and seventeen remain: on those requests, the rule policy and always-dense retrieval each cover seven bundles. The subset of eleven requests labelled non-relation also yields seven successes under each. The larger policy total therefore does not demonstrate that its text-route choices outperform always-dense retrieval. The remaining seventeen still include failed relationship challenges, so the subset is diagnostic rather than a new independently designed test.

An oracle is useful only as a labelled, after-the-fact diagnostic. If it chooses the best result after seeing each request’s judgements, the best of lexical, dense and hybrid covers seven bundles. Allowing it to choose the structured route raises that upper bound to ten, which the rule policy happens to match here. The original trace’s three-route oracle cannot cap a four-route policy. Neither oracle is a deployable, label-free decision rule or evidence that the router will make the same choices on new requests.

Matching that four-route oracle on this constructed sample does not establish correct routing on unfamiliar requests. The inferred intent agrees with the author’s intent label for nineteen of twenty final requests, but those labels encode the same authored design assumptions. The sole mismatch is the request combining checksum and evacuation: the rule calls it terminology and still retrieves its required bundle. Intent-label agreement, route selection and evidence success are three different observations; only the last directly measures the declared retrieval outcome.

Retain the failures that routing cannot repair

Routing does not repair every later-stage failure. The rule policy still misses the maintenance heading-and-action bundle and the long selector tail. A lexical choice triggered by an identifier cannot restore context split across units or make an oversized unit fit the packet. The conflicting-chain request is deliberately refused even though individual positive relationship statements exist: a complete supported path is a stronger obligation than finding several statements that appear relevant in isolation.

In the missing-first-hop case, the policy produces no complete packet while lexical search produces partial evidence. That choice reduces the first-anchor ranking measure, but it does not forfeit a complete bundle that was available for selection. Scoring it as a simple ranking regression, or treating the empty packet as an unqualified success, would erase the distinction between partial evidence and a supported answer path.

The four empty-need requests test rule-based abstention for ambiguity or unsupported relationships. They do not test an abstention decision when an otherwise ordinary information need has no sufficient evidence anywhere in the corpus. Removed or restricted relationship hops test current eligibility instead. A later controller will need genuinely absent-evidence questions and explicit stop reasons that can be reported without leaking protected details.

These outcomes come from a small, authored set of familiar task templates. They establish the execution and diagnostic value of the declared routes, not calibrated confidence, general superiority or reliable behaviour on unrelated requests. No downstream answer consumer was evaluated, so admission, use and helpfulness remain not measured.

A route determines how the system looks; it does not guarantee that the supplied search expression reaches the right vocabulary. Even a correctly selected mechanism may miss evidence when the user’s words differ from the collection’s words. Query Rewriting and Expansion therefore examines a second, separately accountable decision: how to vary a search expression without changing the information need, authority or budget that justified it.

References and further reading

Background and extensions for this chapter: