Parameter-Free Mixture-of-Experts Routing

Follow this topic Ask about this topic

The field studies how to send each token or input to useful experts without relying on a separately learned router, potentially simplifying training and improving expert use. Approaches derive routing signals from hidden-state geometry, online clustering, expert-weight spectra, or fixed category and metadata rules; related methods adjust expert allocation at inference based on confidence or compute budgets. These strategies can reduce load imbalance or inference cost, but their success depends on whether the available signals match the task and the experts’ specializations. The central tension is preserving accuracy and balanced expert use while reducing learned routing, active computation, or supervision, and no single routing rule is reliable across all settings.

Latest papers

13 papers3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Agent-Conditioned Routing: Diverse Reasoning Paths Within One Frozen Model
Oct 27, 2026Ananth Kalyanasundaram, Ankush Kadu, Athira Gopal +1Reasoning Diversity Mixture-of-Experts Language Models
QpiAI, India
Score the Update, Not the Token: Descent-Aligned Routing for Combinatorial LoRA Experts
Sep 30, 2026Priya Nair, Lukas Brenner, Maya Lindqvist +5Mixture-of-Experts Language Models Parameter-Free Mixture-of-Experts Routing
University of California, Irvine · University of Washington
Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models
Sep 29, 2026Yury Nahshan, Nati Daniel, Jacob Goldberger +1Mixture-of-Experts Language Models Parameter-Free Mixture-of-Experts Routing
Bar-Ilan University, Ramat-Gan, Israel · NVIDIA, Israel
MoRE: Scaling mixture of experts with hardware-aware low-rank routing
Sep 28, 2026Honam Wong, Surbhi Goel, Enric Boix-AdseràParameter-Free Mixture-of-Experts Routing Low-Rank Matrix Decomposition
University of Pennsylvania · The Wharton School, University of Pennsylvania
JevSoup: System-One Routing for Training-Free LoRA Composition
Sep 25, 2026Xiuying Wang, Jiahua Cheng, Shuotian Li +4LLM Routing Parameter-Free Mixture-of-Experts Routing
Beijing University of Posts and Telecommunications · University of Malaya · Georgian College +1
Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts
Sep 8, 2026Dohyeon Kim, Bedionita Soro, Sung Ju HwangParameter-Free Mixture-of-Experts Routing Sparse Mixture-of-Experts
KAIST
Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models
Sep 4, 2026Xing Chen, Hengshuai YaoMixture-of-Experts Language Models Mixture-of-Experts Inference
SpecDrop: Parameter-Free Category-Conditioned Routing for Modular Specialization
Aug 4, 2026Boyao Wang, Zhihan LeiParameter-Free Mixture-of-Experts Routing Adaptive Model Routing
Machine Learning Department Carnegie Mellon University
Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA
Jul 28, 2026Tom Saliencro, Rohan Desai, Priya Nair +2Parameter-Free Mixture-of-Experts Routing LLM Inference Acceleration
University of California, Irvine · University of Washington
Hierarchical Copula-Gumbel-Top-\texorpdfstring{ $K$ }{K} Routing: Two-Sided Dependence Control for Frozen Mixture-of-Experts at Fixed Per-Token Routing Laws
Jul 25, 2026Richard Yi Da XuCopula Models Expert Load Balancing
Hong Kong Baptist University and TadReamk Limited
Eigenvectors of Experts are Training-free Non-collapsing Routers
May 29, 2026Giang Do, Hung Le, Truyen TranParameter-Free Mixture-of-Experts Routing Mixture of Experts
Applied Artificial Intelligence Intiative (A2I2), Deakin University, Victoria, Australia.
Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
May 12, 2026Sagi Ahrac, Noya Hochwald, Mor GevaExpert Load Balancing Parameter-Free Mixture-of-Experts Routing
Blavatnik School of Computer Science and AI, Tel Aviv University
Self-Routing: Parameter-Free Expert Routing from Hidden States
Apr 1, 2026Jama Hussein Mohamud, Drew Wagner, Mirco RavanelliParameter-Free Mixture-of-Experts Routing LLM Routing
Mila – Quebec AI Institute · Universite de Montreal · Concordia University

Oct 27, 2026J/K move · Enter open · S save

Agent-Conditioned Routing: Diverse Reasoning Paths Within One Frozen Model

Ananth Kalyanasundaram, Ankush Kadu, Athira Gopal, Aswanth Krishnan

QpiAI, India

Reasoning Diversity Mixture-of-Experts Language Models Parameter-Free Mixture-of-Experts Routing

Open paper AskPDF arXiv

Generating several solutions can expose alternative methods, but additional trajectories often paraphrase one dominant approach. We investigate whether the conditional capacity of a frozen sparse mixture-of-experts (MoE) model can provide such diversity from one problem prompt, without user-assigned roles, prescribed methods, fine-tuning, or model copies. Agent-Conditioned Routing (ACR) assigns

m

parallel trajectories persistent, untrained shared/private expert-eligibility masks while preserving the learned, input-dependent token router. On a screened 126-question multi-mode mathematics and physics benchmark, with model judges reading the reported derivations in full, we count the number of problems yielding at least two distinct primary methods among soft-correct solutions in four trajectories. ACR reaches approximately twice as many such problems as the strongest temperature setting under GLM 5.3 (37.3 versus 18.7), and 1.67 times as many under GPT-5.6 Sol (45.00 versus 27.00), at matched output budgets. A blinded four-expert audit on a stratified sample of 30 problems also finds more multi-method solutions with ACR on both backbones, with lower correctness. Matched-eligibility ablations show modest multi-method count advantages for ACR, while token-resampled masks attain higher correctness. An informed sequential prompt elicits more diversity still, using previous solutions and explicit requests for alternative methods. ACR adds about 160 KiB of mask state. In a three-problem microbenchmark capped at 1,024 tokens, its latency and peak VRAM usage remain within 3% of batched temperature sampling.