Mixture-of-Experts Inference

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, up 167% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 109

All topics

SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs
Jun 3, 2026Yuhao ZhangMixture-of-Experts Pruning LLM Pruning
Beihang University
UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing
Jun 2, 2026Xinming Wei, Chao Jin, Tuo Dai +10Expert Parallelism Expert Load Balancing
School of Computer Science, Peking University · Xiaohongshu Inc. · Shanghai AI Laboratory +1
Beyond Task-Agnostic: Task-Aware Grouping for Communication-Efficient Multi-Task MoE Inference
May 31, 2026Zhiyao Xu, Aoxue Liu, Zhanjie Ding +3Mixture-of-Experts Inference Mixture-of-Experts Models
Tsinghua Shenzhen International Graduate School, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China
ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving
May 30, 2026Seokjin Go, Marko Scrbak, Ephrem Wu +2Expert Load Balancing Mixture-of-Experts Inference
Georgia Institute of Technology Atlanta, GA, USA · Advanced Micro Devices, Inc. Austin, TX, USA · Advanced Micro Devices, Inc. Santa Clara, CA, USA +1
Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference
May 28, 2026Mykola Lukashchuk, Kyrylo Yemets, Wouter M. Kouw +4Factor Graph Optimization Mixture-of-Experts Inference
Eindhoven University of Technology, the Netherlands · Lviv Polytechnic National University, Lviv, Ukraine · Lazy Dynamics, Utrecht, the Netherlands +1
How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
May 27, 2026Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee +9LLM Serving LLM Inference
1Georgia Institute of Technology · 5Infravana · 2Intel +2
MobileMoE: Scaling On-Device Mixture of Experts
May 26, 2026Yanbei Chen, Hanxian Huang, Ernie Chang +5Mixture-of-Experts Language Models On-Device Language Model Inference
Meta AI
ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference
May 26, 2026Xiongwei Zhu, Xiaojian Liao, Tianyang Jiang +3LLM Inference Efficiency Expert Offloading
School of Computer Science and Engineering, Beihang University, Beijing 100191, China · Huawei Technologies Ltd.
Dense2MoE: Pushing the Pareto Frontier of On-Device LLMs via Unified Pruning and Upcycling
May 26, 2026Fengfa Li, Hongjin Ji, Yifeng Ding +2Mixture-of-Experts Pruning LLM Pruning
Li Auto · The Chinese University of Hong Kong, Shenzhen
DepthAgent: Towards Better Universal Depth Estimation via Sample-wise Expert Selection
May 22, 2026Jie Zhu, Girish Chandar Ganesan, Xiaoming LiuMetric Depth Estimation Monocular Depth Estimation
Michigan State University · University of North Carolina at Chapel Hill
Diagnosing Overhead in Dispatch Operations: Cross-architecture Observatory
May 20, 2026Bole Ma, Jan Eitzinger, Harald Koestler +1Expert Parallelism Expert Load Balancing
TIDE: Efficient and Lossless MoE Diffusion LLM Inference with I/O-aware Expert Offload
May 19, 2026Zhiben Chen, Youpeng Zhao, Yang Sui +2Expert Offloading LLM Inference Scheduling
University of Central Florida · Rice University
GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems
May 19, 2026Sourish Wawdhane, Avinash Kumar, Poulami DasExpert Load Balancing Mixture-of-Experts Inference
The University of Texas at Austin
FPED: A Functional-Network Prior-Guided Mixture-of-Experts Framework for Interpretable Brain Decoding
May 19, 2026Yudan Ren, Pengcheng Shi, Zihan Ma +2Neural Decoding Mixture-of-Experts Inference
School of Electronic Information (School of Artificial Intelligence), Northwest University, Xi’an 710127, China
Post-Trained MoE Can Skip Half Experts via Self-Distillation
May 18, 2026Xingtai Lv, Li Sheng, Kaiyan Zhang +12Mixture of Experts Mixture-of-Experts Inference
Tsinghua University · Shanghai AI Lab · WeChat AI +1
CoX-MoE: Coalesced Expert Execution for High-Throughput MoE Inference with AMX-Enabled CPU-GPU Co-Execution
May 18, 2026Muyoung Son, Yi Chen, Seungjae Yoo +2Expert Offloading GPU Acceleration
KAIST · Daejeon, Republic of Korea
BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE
May 14, 2026Juntong Wu, Jialiang Cheng, Qishen Yin +5Sparse Mixture-of-Experts Mixture-of-Experts Inference
Taobao & Tmall Group of Alibaba · Shenzhen Graduate School, Peking University
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
May 12, 2026Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6Compute-in-Memory Mixture-of-Experts Language Models
The Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong · The School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China
Fast MoE Inference via Predictive Prefetching and Expert Replication
May 12, 2026Ankit Jyothish, Ali Jannesari, Aishwarya Sarkar +1Expert Load Balancing GPU Acceleration
Iowa State University Ames, USA
TRACE: Temporal Routing with Autoregressive Cross-channel Experts for EEG Representation Learning
May 12, 2026Fan Ma, Qier An, Peng Chen +6Representation Learning Self-Supervised Pre-Training
Department of Biomedical Informatics and Data Science, Yale University
DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
May 11, 2026Chenyang Song, Weilin Zhao, Xu Han +3Sparse Mixture-of-Experts Mixture-of-Experts Inference
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China
TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling
May 10, 2026Hongyaoxing Gu, Xinzhe Chen, Lijuan Hu +1Efficient Inference Mixture-of-Experts Inference
Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences
Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
May 9, 2026Jongseok Park, Sunga Kim, Zhenyu Gu +2Activation Sparsity Mixture-of-Experts Inference
University of California, Berkeley · Work done during an internship at Advanced Micro Devices, Inc. · Advanced Micro Devices, Inc.
EMO: Pretraining Mixture of Experts for Emergent Modularity
May 7, 2026Ryan Wang, Akshita Bhagia, Sewon MinLanguage Model Pretraining Mixture-of-Experts Language Models
University of California, Berkeley · Allen Institute for AI
Federation of Experts: Communication Efficient Distributed Inference for Large Language Models
May 7, 2026Muhammad Shahir Abdurrahman, Chun Deng, Azalia Mirhoseini +1Expert Parallelism Mixture-of-Experts Inference
Stanford University
Relay Buffer Independent Communication over Pooled HBM for Efficient MoE Inference on Ascend
May 7, 2026Tianlun Hu, Tiancheng Hu, Shengsheng Litang +8Mixture-of-Experts Inference Inference Acceleration
Huawei Technologies
VisMMOE: Exploiting Visual-Expert Affinity for Efficient Visual-Language MoE Offloading
May 7, 2026Cheng Xu, Xiaofeng Hou, Jiacheng Liu +1Expert Offloading Vision-Language Models
Shanghai Jiao Tong University Minhang Qu, Shanghai, China
Adaptive Inverted-Index Routing for Granular Mixtures-of-Experts
May 6, 2026Klaus-Rudolf Kladny, Maximilian Mordig, Bernhard Schölkopf +1Mixture-of-Experts Inference Expert Routing
Max Planck Institute for Intelligent Systems, Tübingen · Tübingen AI Center · ETH Zurich +1
MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
May 3, 2026Zhaoyuan Su, Olatunji Ruwase, Karthik Ganesan +5Expert Parallelism LLM Serving
University of Virginia · 2Snowflake AI Research · 3Harvard University
Affinity Is Not Enough: Recovering the Free Energy Principle in Mixture-of-Experts
May 1, 2026Man Yung WongSparse Mixture-of-Experts Mixture-of-Experts Inference
Russell · Independent Researcher

Jun 3, 2026, cs.LGJ/K move · Enter open · S save

SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs

Yuhao Zhang

Beihang University

Mixture-of-Experts Pruning LLM Pruning Sparse Mixture-of-Experts Mixture-of-Experts Inference Mixture-of-Experts Models

Open paper AskPDF arXiv

Sparse Mixture-of-Experts (MoE) large language models achieve strong quality with low per-token compute, yet their deployment is often limited by the memory wall: the full expert pool must remain resident to support token-dependent routing. Expert pruning is a direct remedy, but prior criteria typically score experts independently and overlook that MoE inference is inherently \emph{coalitional}, where outputs arise from routed top-

k

expert combinations. We propose \textbf{SHAPE}, a task-driven pruning framework that explicitly models \emph{intra-layer} expert cooperation. SHAPE formulates routing traces on a small calibration set as an empirical cooperative game and assigns interaction-aware expert values via a Shapley-style attribution over observed top-

k

coalitions, enabling the identification of experts that are essential for high-utility collaborations rather than merely frequent. To preserve MoE topology under a global pruning budget, SHAPE further introduces a \emph{quality-coverage} selection rule that retains, in each layer, the minimal expert subset covering an

α

fraction of non-negative Shapley mass, while using bisection to match a target keep rate. Experiments on three modern MoE backbones (Qwen3-30B-A3B, GPT-OSS-20B, and DeepSeek-V2-Lite) across diverse benchmarks show that SHAPE consistently improves robustness over global and layer-wise pruning variants, maintaining competitive accuracy under 20% and 40% expert pruning without additional training and delivering clear reductions in peak GPU memory footprint. The open-source code is available at https://github.com/Alizen-1009/Shapley-Moe.

Momentum

Related topics

Latest papers 109