Routing-Free Mixture-of-Experts

Yilun Liu; Jinru Han; Sikuan Yan; Volker Tresp; Yunpu Ma

cs.LGApr 1, 2026

Routing-Free Mixture-of-Experts

Authors: Yilun Liu, Jinru Han, Sikuan Yan, Volker Tresp, Yunpu Ma

Organizations: Ludwig Maximilian University of Munich · Munich Center for Machine Learning · University of California, Los Angeles

Abstract

Standard Mixture-of-Experts (MoE) models rely on centralized routing mechanisms that introduce rigid inductive biases. We propose Routing-Free MoE which eliminates any hard-coded centralized designs including external routers, Softmax, Top-K and load balancing, instead encapsulating all activation functionalities within individual experts and directly optimized through continuous gradient flow, enabling each expert to determine its activation entirely on its own. We introduce a unified adaptive load-balancing framework to simultaneously optimize both expert-balancing and token-balancing objectives through a configurable interpolation, allowing flexible and customizable resource allocation. Extensive experiments show that Routing-Free MoE can consistently outperform baselines with better scalability and robustness. We analyze its behavior in detail and offer insights that may facilitate future MoE design ad optimization.

Figures & tables

Figure 1: Standard MoE relies on routing to orchestrate expert activations. Routing-Free MoE let each expert purely-independently determine its own activation. Green indicates activated components; red for inactive components; yellow for trainable components.

Figure 2: Routing-Free MoE consistently outperforms standard MoE, AoE ( Lv et al., 2025 ) , and ReMoE ( Wang et al., 2024b ) in language modeling. All models are trained on OpenWebText ( Gokaslan et al., 2019 ) under identical environment conditions and best-performing configurations, as described in Section 4.1 . FLOPs are estimated for one epoch.

Panel of Figure 3 — Figure 3: Load-balancing for tokens and experts. Routing-Free MoE introduces a unified load-balancing framework that simultaneously optimizes both expert-balancing and token-balancing through a configurable interpolation.

Panel of Figure 4 — Figure 4: Training dynamics of Routing-Free MoE at scale S, with $r=16$ , $\lambda_{0}=1\mathrm{e}^{-10}$ , $\eta=0.02$ , and $\alpha=1\mathrm{e}^{-3}$ .

Config.	Size	FLOPs	PPL $\downarrow$
Standard MoE	92.44M	90.93M	31.22
w/o router (AoE, $r$ =16)	93.85M	88.57M	30.00
w/o router (AoE, $r$ =32)	95.32M	91.08M	30.31
w/o TopK&Softmax (ReMoE)	92.44M	90.93M	29.60
Routing-Free MoE ( $r$ =16)	93.85M	88.57M	28.73
Routing-Free MoE ( $r$ =32)	95.32M	91.08M	28.33

Table 2: Ablation performed at scale S. Detailed configurations for each run are provided in Table E .

$\bm{r}$	Size	FLOPs	PPL $\downarrow$
8	93.11M	87.32M	29.16
16	93.85M	88.57M	28.74
32	95.32M	91.08M	28.34
64	98.27M	96.09M	28.24

Table 3: Effect of low-rank projection dimension at scale S, with learning rate

\alpha=1\mathrm{e}^{-3}

Panel of Figure 5 — Figure 5: Training dynamics of Routing-Free MoE by $\alpha$ , with $r=16$ , $\lambda_{0}=1\mathrm{e}^{-10}$ , and $\eta=0.02$ .

Figure 6: Evolution of expert activation across layers during training on OpenWebText under global density target. Left panel shows the per - layer mean activation as lines, along with IQR as shaded band regions using 1,000 - step moving average smoothing. Right panel shows activation distribution at the final training step. Model at scale S.

	$\bm{\mu}$	PPL $\downarrow$	Eval. Throughput $\uparrow$
only TB	0.0	28.41	645.7
	0.2	28.35	648.3
balanced	0.5	28.34	662.3
	0.8	28.38	643.9
only EB	1.0	28.43	648.8

Table 4: Effect of load balancing interpolation

\mu

. Throughput is measured as samples per second.

Panel of Figure 7 — Figure 7: Expert activation heatmaps on different input for representative layers at scale S. Each row shows the average expert activation ratios on a given benchmark. Darker colors indicate more frequent activation.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	$\bm{T}$	$\bm{M}$	$\bm{K}$	Throughput $\uparrow$
Prefill Stage
MoE	1,024	1	3	22,346.66
MoE	1,024	2	3	18,034.84
MoE	1,024	3	3	18,558.08
MoE	1,024	4	3	18,355.43
RFMoE	1,024	1	3	22,277.77

Appendix

Table 5: Prefill and autoregressive decode tokens-per-second throughput per device under expert parallelism. Prefill process

T

=1,024 tokens in a single forward pass; decode generates 128 tokens autoregressively with

T

=1 per step.

K

=3 denotes TopK for standard MoE and

K_{\text{eff}}

=3 for Routing-Free MoE. All runs use batch size 1 with input tokens sampled randomly from OpenWebText, bfloat16 precision, and report the mean over 10 repetitions after 3 warmup ones.

$\bm{\theta}$	$\bm{\rho}_{\text{eff}}$ (%)	FLOPs	Avg. $\uparrow$
0.1	100.0	120.46M	39.41
0.5	94.8	118.41M	39.12
0.8	61.4	105.40M	39.55
0.9	48.9	100.32M	39.73
1.0	37.8	96.21M	39.80
1.1	29.3	92.97M	39.98

Appendix

Table 6: Effect of the global threshold

\theta

on downstream benchmark average (Avg.) and estimated FLOPs at scale S.

\rho_{\text{eff}}

denotes the empirical mean activation density at

\theta

. † Averages at

\theta\geq 1.5

are driven by the anomalous SST-2 and OBQA spikes (see Figure 8 ).

Figure 8: Per-benchmark accuracy across

\theta

at scale S. HellaSwag, QQP, QNLI are nearly invariant to the threshold, while PIQA, ARC-easy, ARC-challenge and Winogrande achieve their best performance around

\rho_{\text{eff}}\approx\rho_{\infty}

. SST-2 and OBQA spikes sharply at larger

\theta

, driving the elevated averages.

Scale	W/L	$\Delta$ Avg.	$\bm{t}$ -stat	$\bm{p_{t}}$	Cohen’s $\bm{d}$
S	5 / 4	+0.80	1.481	0.089	0.494
M	6 / 3	+0.76	0.764	0.233	0.255
L	5 / 4	+0.76	1.184	0.135	0.395
Overall	16 / 11	+0.77	1.858	0.037	0.358

Appendix

Table 7: Paired statistical tests comparing Routing-Free MoE against the standard MoE baseline across 9 benchmarks.

p

-values are one-sided.

Mixture-of-Experts Models

Mixture-of-experts models use input-dependent routing to activate specialized subnetworks, increasing model capacity without running every parameter for each input.

See all 433 papers

ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
Jun 1, 2026Heng Zhao, Zilei Shao, Guy Van den Broeck +1Sparse Mixture-of-Experts Expert Routing
Department of Computer Science, University of Virginia, Charlottesville, USA · Department of Computer Science, University of California, Los Angeles, Los Angeles, USA
Three Phases of Expert Routing: How Load Balance Evolves During Mixture-of-Experts Training
Apr 5, 2026Charafeddine MouzouniExpert Load Balancing Mixture-of-Experts Language Models
OPIT – Open Institute of Technology, and Cohorte AI, Paris, France.
Self-Routing: Parameter-Free Expert Routing from Hidden States
Apr 1, 2026Jama Hussein Mohamud, Drew Wagner, Mirco RavanelliParameter-Free Mixture-of-Experts Routing LLM Routing
Mila – Quebec AI Institute · Universite de Montreal · Concordia University

Jun 1, 2026J/K move · Enter open · S save

ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts

Heng Zhao, Zilei Shao, Guy Van den Broeck, Zhe Zeng

Department of Computer Science, University of Virginia, Charlottesville, USA · Department of Computer Science, University of California, Los Angeles, Los Angeles, USA

Sparse Mixture-of-Experts Expert Routing Mixture-of-Experts Models

Open paper AskPDF arXiv

Mixture-of-Experts (MoE) models scale by activating only a small subset of experts per token. However, training such models remains challenging because top-

k

routing is discrete and non-differentiable, requiring gradient estimators for expert selection whose design remains a central open problem. We introduce ProbMoE, a probabilistic routing framework that models expert selection as a distribution over cardinality-constrained expert subsets and formulates routing as probabilistic inference in this discrete subset space. We first propose ProbMoE Exact-

k

routing, which samples

k

-expert subsets in the forward pass, and the backward pass uses gradients through each expert's exact marginal probability as a tractable surrogate for the true gradient. ProbMoE naturally generalizes to a dynamic-

k

routing setting, where both training and inference constrain the routing cardinality to the same predefined range, allowing adaptive expert allocation per token. Across benchmarks and model backbones, ProbMoE Exact-

k

achieves strong performance compared to competitive baselines, with improved expert utilization and routing diversity; ProbMoE Dynamic-

k

achieves comparable performance with fewer activated experts.

Abstract

Mixture-of-Experts Models

Explore similar work

ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts

Three Phases of Expert Routing: How Load Balance Evolves During Mixture-of-Experts Training

Self-Routing: Parameter-Free Expert Routing from Hidden States