cs.CLJan 2, 2026

Embedding Consistency Regulation for Persistent Semantic Routing in Compact Language Models

Authors: Chung-Wei Victor Yuan

Organizations: YVIC Research Lab

Abstract

Language models are increasingly deployed under tight memory and compute constraints, where semantic routes (e.g., language, tone, or intent) may be assigned before decoding. However, these routes may not remain reflected in generated responses. We study this phenomenon as persistent semantic routing. We investigate this property using Embedding Consistency Regulation (ECR) as a minimal probe. ECR derives a short discrete prefix for each input using a frozen embedding model and applies the same prefix construction during both training and inference. In controlled prefix-assignment experiments, ECR achieves 99.93% ±\pm 0.12% language routing accuracy, while random, shuffled, and dummy controls remain near chance (34--36%). This gap is not reliably captured by likelihood: some controls achieve competitive NLL despite losing routing consistency. We introduce Response Routing Recoverability (RRR), an encoder-independent diagnostic that tests whether assigned routes remain observable after generation. Routing recoverability persists across multilingual settings, unsupervised partitions, reduced precision, and transfer tasks. These findings establish routing persistence as a distinct and measurable property of language models, complementary to task performance and likelihood.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Oct 9, 2026

CopRoute: Dependence-Aware Conformal Routing for Language Models

Set-based language model routing aims to increase the coverage of the event where at least one model answers a query correctly while controlling the number of inference calls made. Existing risk-controlled routing policies bound the all-miss risk (the probability that every model in the selected set fails) by calibrating a single conformal threshold and applying it to a nested family of model sets. Because each model is scored in isolation, these policies ignore the dependence structure of model failures. Models that share training data or architecture tend to fail on the same queries, so two individually strong models can be jointly redundant while a weaker but complementary model covers the queries that the others miss. To account for this dependence, which existing risk-controlled policies ignore, we propose CopRoute, which orders models by how many new queries each one answers correctly beyond those already selected, decides per query how far down that order to go, and then picks the smallest cutoff that still meets the target risk. We show that CopRoute's ordering attains, on the design data, the best guarantee any efficient fixed-ordering algorithm can give for the average number of calls to the first correct model, and that a separate calibration step certifies the all-miss risk of the deployed sets. On RouterBench and RouterEval, CopRoute attains the fewest certified calls on nine of ten datasets at a target risk of 0.05 and on eight of ten at 0.10, making up to 79 percent fewer calls (median 24 percent) than the state-of-the-art risk-controlled policy at the same certified all-miss risk.
Aug 4, 2026

Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages

When a language model follows an in-context conditional rule such as "if P(x) then A else B," does it assemble a runtime circuit with one module that tests the predicate and another that routes the answer? We probe this with activation patching under a four-donor design whose two swapped-rule donors make the condition and the answer word disagree, so each layer reveals which of the two it carries. Across three open models from two families and six languages sharing one fixed item bank, a mid-stack residual band carries the predicate's truth value: patching it reroutes the answer with predicate-outcome flip near 1.0 and mapping flip near 0.0, meeting a strict pre-specified isolation criterion in 17 of 18 cells, and the same localization holds across five predicate families. The router shows the opposite profile. A learned subspace flips A and B near-perfectly within the trained pair yet transfers to a new pair at approximately 0 in every model, while in Gemma-3-4B (the only model probed cross-lingually) it transfers at approximately 0.98 to the same pair in other languages. Under every probe we ran, the router direction is token-bound and non-transferable (largely answer-readout in Gemma, pair-specific in Qwen) rather than an abstract routing module. Test is modular; under these probes, route is not.
Sep 28, 2026

SeLMRoute: Probabilistic Semantic Evidence for Large Language Model Routing

Large language model (LLM) routing aims to select the most suitable model for each incoming query. Most existing routers learn this decision directly from query embeddings, model representations, preference data, or clusters of similar examples. Such approaches can be effective, yet the representation used for routing rarely states what a query actually requires. We introduce SeLMRoute, a routing framework that separates the extraction of candidate-independent semantic evidence from the learning of candidate performance and the application of deployment objectives. A decision model first evaluates a set of interpretable questions about the query, such as its reasoning requirements and use of external knowledge, with each judgment retained as a probability distribution. The resulting probabilistic semantic state is used by a lightweight supervised router to estimate candidate model performance. Routing objectives are applied after performance estimation, which allows the same semantic state to support performance-oriented and cost-aware decisions. On the LLMRouterBench (15 datasets, 20 candidate models, 11,481 queries), SeLMRoute achieves an average accuracy of 72.08%±0.4572.08\% \pm 0.45, while grouped five-fold out-of-fold evaluation reaches 72.64%72.64\%, compared with 69.23%69.23\% for the strongest fixed candidate. The representation achieves the highest mean performance among the evaluated semantic, dense, lexical, and domain-level representations. In a separate 13-model performance-cost setting, SeLMRoute improves performance in all five grouped splits, with a mean PerfGain of 2.66%2.66\%. Our code is available at https://github.com/Indigma-Innovations/SeLMRoute.