Efficient expert routing is a fundamental challenge in Mixture-of-Experts (MoE) systems, particularly for heterogeneous collections of independently trained Large Language Mod els (LLMs). Existing routing approaches typically rely on jointly trained gating networks, prompt-based predictors, or computationally expensive multi-model inference, limiting their flexibility and efficiency. This paper presents ScoutRoute, a lightweight post-hoc routing framework that predicts expert competence using internal language model signals extracted from a single forward pass through an inexpensive scout model. Hidden-state representations are used to estimate cross-model competence through lightweight logistic probes, while logit-derived confidence measures are incorporated within a domain-aware routing strategy for efficient expert selec tion without modifying the underlying language models. ScoutRoute isevaluatedon fiveopen-weightlanguagemodels across fourbenchmark datasets covering mathematical reasoning, code generation, commonsense reasoning, and multidis ciplinary knowledge. Experimental results show that ScoutRoute consistently outperforms individual routing signals while requiring only a single scout-model inference. The proposed competence probes also achieve superior calibration compared with conventional confidence measures, demonstrating that internal language model representations provide transferable information for accurate and efficient expert routing across heterogeneous language models
Figures & tables
Figure 1: Overall architecture of the proposed ScoutRoute framework. The figure illustrates the single-pass routing pipeline, including scout model inference, internal signal extraction, competence estimation, domain-aware routing, and expert selection.
Routing Strategy
Accuracy
Random Routing
35.36%
Negative Entropy
63.47%
Probe Score
63.16%
Mean Log Probability
65.02%
Oracle Router
82.66%
Table 1: Routing accuracy of standalone routing strategies.
Routing Method
Accuracy
Random Routing
35.54%
Negative Entropy
63.47%
Probe Score
63.16%
Mean Log Probability
65.02%
ScoutRoute (Domain-Aware)
67.80%
Oracle Router
82.66%
Table 2: Comparison of standalone routing signals and the proposed ScoutRoute framework.
Scout Model
AUROC
Qwen2.5-1B
0.8957
Phi-3 Mini
0.8610
Qwen2.5-3B
0.8231
Qwen2.5-Math
0.7240
Qwen2.5-Coder
0.6823
Table 3: Competence probe performance measured using AUROC.
Scout Model
Log Prob.
Neg. Entropy
Probe
Qwen2.5-1B
0.4691
0.4299
0.1456
Qwen2.5-3B
0.4570
0.4548
0.1289
Phi-3 Mini
0.4723
0.3155
0.1313
Qwen2.5-Math
0.4155
0.3236
0.2313
Qwen2.5-Coder
0.3632
0.3868
0.2433
Table 4: Expected Calibration Error (ECE) of different routing signals. Lower values indicate better calibration.
Scout Model
Probe Router
ScoutRoute
Qwen2.5-1B
65.33%
67.80%
Qwen2.5-3B
62.85%
66.87%
Phi-3 Mini
65.33%
68.42%
Qwen2.5-Math
58.51%
65.33%
Qwen2.5-Coder
61.92%
67.18%
Table 5: Routing accuracy obtained using different scout models.
Domain
Specialist
Routed (%)
Specialist Acc.
Defection Acc.
GSM8K
Qwen2.5-Math
72.86
86.27
73.68
HumanEval
Qwen2.5-Coder
72.17
67.47
15.62
Table 6: Analysis of specialist routing behaviour.
Large language model routing selects models, tools, experts, or inference strategies to balance answer quality, cost, latency, and deployment constraints.
Mixture-of-Experts (MoE) language models route each token to a small subset of experts, but whether the routes selected by a trained top-k router are good ones is rarely evaluated directly. Holding the model fixed, we compare each standard route against sampled equal-compute alternatives for the same token and score each by the next-token probability it assigns to the realized token in a verified reasoning trajectory. The result is sharply token-conditional: the standard router is well-aligned with route utility on confident tokens but uninformative on the fragile tokens that drive hard reasoning, where lower-loss equal-compute routes consistently exist inside the frozen model but are not selected. The same pattern holds across Qwen3-30B-A3B, GPT-OSS-20B, DeepSeek-V2-Lite, and OLMoE-1B-7B, and follows structurally from how standard top-k training evaluates routing decisions: the language modeling loss scores only the executed route, and load balancing depends only on aggregate routing statistics. A minimal router-only update to the final-layer router, leaving every expert and every other router frozen, is sufficient to shift pass@K on AIME 2024+2025 and HMMT 2025 for both Qwen3-30B-A3B and GPT-OSS-20B, suggesting that at least part of the failure reflects router-reachable misallocation rather than expert capacity alone.
Youngsik Yoon, Siwei Wang, Wei Chen +1
Department of Computer Science and Engineering, POSTECH, South Korea · Microsoft Research Asia, Beijing, China · Graduate School of Artificial Intelligence, POSTECH, South Korea
Sparse Mixture-of-Experts (MoE) architectures are increasingly popular for frontier large language models (LLM) but they introduce training challenges due to routing complexity. Fully leveraging parameters of an MoE model requires all experts to be well-trained and to specialize in non-redundant ways. Assessing this, however, is complicated due to lack of established metrics and, importantly, many routing techniques exhibit similar performance at smaller sizes, which is often not reflective of their behavior at large scale. To address this challenge, we propose the MoE Routing Testbed, a setup that gives clearer visibility into routing dynamics at small scale while using realistic data. The testbed pairs a data mix with clearly distinguishable domains with a reference router that prescribes ideal routing based on these domains, providing a well-defined upper bound for comparison. This enables quantifiable measurement of expert specialization. To demonstrate the value of the testbed, we compare various MoE routing approaches and show that balancing scope is the crucial factor that allows specialization while maintaining high expert utilization. We confirm that this observation generalizes to models 35x larger.
Mixture-of-Experts (MoE) architectures enable efficient model scaling, yet expert routing behavior across underrepresented languages remains poorly understood. We analyze routing dynamics in two architecturally distinct MoE models -- a pure Transformer (Qwen3-30B-A3B) and a hybrid Mamba-Transformer (Nemotron-3-Nano-30B-A3B) -- using Hebrew as a morphologically rich, low-resource testbed. Both pre-trained models exhibit deep-layer routing collapse: usage entropy drops sharply in final layers and tokens concentrate on a narrow expert subset, a pattern largely absent for English. Continual pre-training (CPT) on balanced bilingual data substantially corrects this imbalance, increasing entropy and shifting routing toward shared, language-agnostic experts; supervised fine-tuning (SFT) alone achieves less complete correction. Extending the analysis to Japanese reveals quantitatively consistent collapse signatures, providing cross-linguistic evidence that the phenomenon is a systematic consequence of pre-training underrepresentation rather than any language-intrinsic property. Routing improvements correlate with consistent downstream benchmark gains, positioning routing entropy and expert specialization as principled diagnostics for multilingual capacity in MoE systems.