Expert Routing

Follow this topic Ask about this topic

Expert routing studies how models assign tokens, examples, or tasks to a small subset of experts, enabling conditional computation while storing many specialized networks. Common approaches include top-k token-choice and expert-choice gates, hierarchical and threshold-based routing, and adaptive policies driven by uncertainty, task information, or learned structure. Researchers test these methods in autoregressive language models, vision and multimodal systems, and diffusion models, measuring quality on benchmarks such as WikiText, MMLU, GSM8K, and ImageNet alongside load balance, latency, and memory use. A central tension is that balancing expert workloads can conflict with specialization and efficient execution, while analyses increasingly question whether routing patterns alone demonstrate genuine domain expertise or useful downstream effects.

Latest papers

119 papers29 papers in the last four weeks, up 190% on the four weeks before. 0.2% of all new papers.

Adaptive Inverted-Index Routing for Granular Mixtures-of-Experts
May 6, 2026Klaus-Rudolf Kladny, Maximilian Mordig, Bernhard Schölkopf +1Mixture-of-Experts Inference Expert Routing
Max Planck Institute for Intelligent Systems, Tübingen · Tübingen AI Center · ETH Zurich +1
Soft-to-Hard Routing in Sparse Mixture-of-Experts Models
May 4, 2026Reza RastegarSparse Mixture-of-Experts Expert Routing
Meta Platforms, Inc Woodinville, WA
Affinity Is Not Enough: Recovering the Free Energy Principle in Mixture-of-Experts
May 1, 2026Man Yung WongSparse Mixture-of-Experts Mixture-of-Experts Inference
Russell · Independent Researcher
SceneSelect: Selective Learning for Trajectory Scene Classification and Expert Scheduling
Apr 27, 2026Xinrun Wang, Deshun Xia, Yuxi Sun +1Multi-Agent Trajectory Prediction Expert Routing
School of Computer Science, China University of Geosciences (Wuhan) · School of Information Engineering, Wuhan University of Technology · School of Mathematics and Statistics, Wuhan University of Technology
SMoES: Soft Modality-Guided Expert Specialization in MoE-VLMs
Apr 27, 2026Zi-Hao Bo, Yaqian Li, Anzhou Hou +6Vision-Language Models Efficient VLM Inference
Li Auto Inc.
Teacher-Guided Routing for Sparse Vision Mixture-of-Experts
Apr 23, 2026Masahiro Kada, Ryota Yoshihashi, Satoshi Ikehata +2Adaptive Model Routing Sparse Mixture-of-Experts
Institute of Science Tokyo · Tokyo, Japan · DENSO IT Laboratory
Multi-Domain Learning with Global Expert Mapping
Apr 20, 2026Pourya Shamsolmoali, Masoumeh Zareapoor, Huiyu Zhou +3Vision Foundation Model Adaptation Expert Routing
Department of Computer Science, University of York, UK · School of Computing and Mathematical Sciences, University of Leicester, UK · Centre for Vision Speech and Signal Processing, University of Surrey, UK +2
Layer-wise MoE Routing Locality under Shared-Prefix Code Generation: Token-Identity Decomposition and Compile-Equivalent Fork Redundancy
Apr 19, 2026Shun-ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino +1Expert Routing Efficient Language Model Inference
Graduate School of Informatics, Nagoya University, Japan · Information Technology Center, Nagoya University, Japan
CoGR-MoE: Concept-Guided Expert Routing with Consistent Selection and Flexible Reasoning for Visual Question Answering
Apr 18, 2026Xiyin Zeng, Yi Lu, Hao WangVisual Question Answering Expert Routing
Hong Kong University of Science and Technology (Guangzhou)
Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality
Apr 15, 2026Ivan Ternovtsii, Yurii BilakMixture-of-Experts Language Models Expert Routing
Department of Software Systems, Uzhhorod National University Narodna sq. 3, Uzhhorod, Ukraine, 88000 · HengeBytes
The Myth of Expert Specialization in MoEs: Why Routing Reflects Geometry, Not Necessarily Domain Expertise
Apr 10, 2026Xi Wang, Soufiane Hayou, Eric NalisnickMixture-of-Experts Expert Specialization Mixture of Experts
Johns Hopkins University
Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
Apr 9, 2026Haolei Xu, Haiwen Hong, Hongxing Li +7VLM Reasoning Expert Routing
Zhejiang University · Alibaba Group
MoE Routing Testbed: Studying Expert Specialization and Routing Behavior at Small Scale
Apr 8, 2026Tobias Falke, Nicolas Anastassacos, Samson Tan +6Mixture of Experts Sparse Mixture-of-Experts
Amazon AGI
Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models
Apr 2, 2026Shuibai Zhang, Caspian Zhuang, Chihan Cui +8Expert Routing Mixture-of-Experts Language Models
University of Wisconsin-Madison · Scitix · Cornell University +4
MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models
Mar 26, 2026Dohwan Ko, Jinyoung Park, Seoung Choi +3Expert Routing Mixture-of-Experts Models
Korea University · KAIST
LGEST: Dynamic Spatial-Spectral Expert Routing for Hyperspectral Image Classification
Mar 25, 2026Jiawen Wen, Suixuan Qiu, Zihang Luo +2Hyperspectral Image Classification Hybrid CNN-Transformer Architectures
Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China · Faculty of Geographical Science, Beijing Normal University, Beijing, 100875, China · School of Electronic and Communication Engineering, Guangzhou University, Guangzhou University, Guangzhou, Guangzhou, 510006, China
MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
Mar 25, 2026Andrea ManzoniMixture-of-Experts Fine-Tuning Fine-Tuning
Path-Constrained Mixture-of-Experts
Mar 18, 2026Zijin Gu, Tatiana Likhomanenko, Vimal Thilak +2Mixture-of-Experts Language Models Sparse Mixture-of-Experts
Apple · †Google
Expert Threshold Routing for Autoregressive Language Modeling with Dynamic Computation Allocation and Load Balancing
Mar 12, 2026Hanchi Sun, Yixin Liu, Yonghui Wu +1Expert Routing Mixture-of-Experts Models
Computer Science and Engineering, Lehigh University, Bethlehem, PA, USA · MD-HOBI-BIOMED INFORMATICS, University of Florida, Gainesville, FL, USA
Task-Conditioned Routing Signatures in Sparse Mixture-of-Experts Transformers
Mar 11, 2026Mynampati Sri Ranganadha AvinashMixture-of-Experts Interpretability Mixture-of-Experts Language Models
Independent Researcher | Asthra Labs
Mixture of experts architectures for machine learning interatomic potentials
Mar 9, 2026Yuzhi Liu, Duo Zhang, Anyang Peng +3Machine Learning Interatomic Potentials Mixture-of-Experts Models
AI for Science Institute, Beijing 100080, P. R. China. · DP Technology, Beijing 100080, P. R. China. · Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, P. R. China. +3
DynaMoE: Dynamic Token-Level Expert Activation with Layer-Wise Adaptive Capacity for Mixture-of-Experts Neural Networks
Mar 2, 2026Gökdeniz GülmezMixture-of-Experts Models Expert Routing
Machine Learning Research, Stuttgart, Germany
MME: Mixture of Mesh Experts with Random Walk Transformer Gating
Feb 28, 2026Amir Belder, Ayellet TalMixture-of-Experts Models Expert Routing
Technion - Israel Institute of Technology, Israel
Grouter: Decoupling Routing from Representation for Accelerated MoE Training
Feb 22, 2026Yuqi Xu, Rizhen Hu, Zihan Liu +2Mixture-of-Experts Models Expert Routing
School of Mathematical Sciences, Peking University, Beijing, China · Zhejiang Lab, Hangzhou, China · Yuanpei College, Peking University, Beijing, China +1
Grassmannian Mixture-of-Experts: Concentration-Controlled Routing on Subspace Manifolds
Feb 19, 2026Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaExpert Routing Mixture-of-Experts Language Models
Department of Computer Science, Iowa State University · Department of Civil, Construction and Environmental Engineering, Iowa State University
Phase-Aware Mixture of Experts for Agentic Reinforcement Learning
Feb 19, 2026Shengtian Yang, Yu Li, Shuo He +4Reinforcement Learning RL for LLM Agents
School of Computer Science and Engineering, Southeast University, Nanjing, China · Kuaishou Technology, Beijing, China · College of Computing and Data Science, Nanyang Technological University, Singapore
OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
Feb 5, 2026Jingze Shi, Zhangyang Peng, Yizhang Zhu +3Mixture-of-Experts Inference LLM Inference Acceleration
The Hong Kong University of Science and Technology (Guangzhou) · Beijing Academy of Artificial Intelligence
Routing-Aware Safety Alignment for Mixture-of-Experts Models
Feb 4, 2026Jiacheng Liang, Yuhui Wang, Tanqiu Jiang +1Mixture-of-Experts Language Models LLM Alignment
Department of Computer Science Stony Brook University
L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
Jan 29, 2026Minghao Yang, Ren Togo, Guang Li +2Expert Routing Mixture-of-Experts Models
Hokkaido University

May 6, 2026J/K move · Enter open · S save

Adaptive Inverted-Index Routing for Granular Mixtures-of-Experts

Klaus-Rudolf Kladny, Maximilian Mordig, Bernhard Schölkopf, Michael Muehlebach

Max Planck Institute for Intelligent Systems, Tübingen · Tübingen AI Center · ETH Zurich+1

Mixture-of-Experts Inference Expert Routing Mixture-of-Experts Models

Open paper AskPDF arXiv

Mixture-of-experts (MoE) models enable scalable transformer architectures by activating only a subset of experts per token. Recent evidence suggests that performance improves with increasingly granular experts, i.e., many small experts instead of a few large ones. However, this regime substantially increases routing cost, which can dominate computation. We introduce adaptive inverted-index routing for MoE (AIR-MoE), an inverted-index-inspired routing architecture based on vector quantization (VQ). In a first stage, AIR-MoE performs coarse shortlisting by assigning tokens to VQ codewords to construct a candidate set of experts. In a second stage, fine scoring computes exact routing scores restricted to this shortlist. This two-stage procedure approximates true top-k routing while avoiding full expert scoring and, in contrast to prior work, imposing no structural constraints on expert parameters. AIR-MoE serves as a drop-in replacement for standard routers and requires no modifications to the model architecture or loss function. We further provide a lower bound on the mass recall achieved by AIR-MoE that yields insights into its inner workings. Empirically, we demonstrate that AIR-MoE achieves improved performance compared to existing routing approaches in granular MoE settings.