Mixture-of-Experts Pruning

Follow this topic Ask about this topic

Mixture-of-Experts pruning studies how to shrink models made of specialized subnetworks while retaining their accuracy and efficiency advantages. Methods rank experts using signals such as routing frequency, activation strength, weight or gradient sensitivity, and estimated deletion error; others account for expert interactions or allocate pruning budgets across layers. Approaches range from training-free expert removal and neuron-level trimming to merging, distillation, and dynamic selection at inference, with evaluation spanning language, reasoning, coding, and domain-specific tasks. A central challenge is that importance estimates and safe pruning levels vary by model, task, and deployment setting, so gains in memory or speed must be weighed against capability loss and changes in generation behavior.

Latest papers

59 papers17 papers in the last four weeks, up 70% on the four weeks before. 0.1% of all new papers.

CRISP: Co-designing Pruning and Scheduling for Efficient MoE Inference on Edge Servers
Oct 27, 2026Congwei Zhang, Youming Tao, Yifei Zou +6Mixture-of-Experts Inference Memory-Efficient Inference
School of Electrical Engineering and Computer Science, TU Berlin, Berlin, Germany · School of Computer Science and Technology, Shandong University, Qingdao, China · Department of Data Science, City University of Hong Kong, Hong Kong, China
GS-MoE: Training-Free MoE Compression via Groupwise Subspace Sharing and Pruning
Oct 23, 2026Carson Cary, Yukai Jin, Yihao Huang +5Mixture-of-Experts Models Mixture-of-Experts Pruning
Florida State University · University of Pennsylvania · University of Notre Dame
ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models
Oct 22, 2026Jiaqi Chen, Xianglong Yan, Chengzhu Bao +2Mixture-of-Experts Quantization Post-Training Quantization
Shanghai Jiao Tong University
SPARE: Sparsity Allocation via Routing Entropy for Efficient MoEs
Oct 21, 2026Yifan Yang, Hanling Yi, Dongquan Yang +4Mixture-of-Experts Pruning Sparse Mixture-of-Experts
Intellifusion Inc.
Don't Skip the Baseline: A Controlled Comparison of Dynamic Expert Skipping in MoEs
Oct 20, 2026Jakob Steimle, Thomas Elsken, Daniel Ehrhardt +3Mixture-of-Experts Inference Mixture-of-Experts Pruning
Technical University of Munich · Tensordyne · ETH Zurich +1
Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models
Oct 19, 2026Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin +7Mixture-of-Experts Pruning Mixture-of-Experts Language Models
LinkedIn · MIT · Google
Fine-Grained Units, Flat Decisions: Hierarchical Micro-Expert Pruning for Mixture-of-Experts Models
Oct 19, 2026Zukang Xu, Xinpei Gao, Xing Hu +2Channel Pruning Mixture-of-Experts Pruning
Houmo AI · University of Science and Technology of China · Southeast University
PW-Pruning: Phase-Aware Expert Selection for Speech-Centric Omni-Modal Models
Oct 15, 2026Junjie Cao, Fengping Tian, Wenjie Zhou +5Mixture-of-Experts Pruning Mixture-of-Experts Models
Alibaba, Hangzhou, China · University of Macau
Towards Robust Calibration Dataset for Domain-General Mixture of Experts Pruning
Oct 11, 2026Luis Frentzen Salim, Chia-Chun Wu, Tran Van Nhiem +2Mixture-of-Experts Pruning Neural Network Pruning
Institute of Information Science, Academia Sinica · National Taiwan University of Science and Technology · AI Research Center, Hon Hai Research Institute
SPEAR: A Spectral-Disentangled MoE Neural Operator with Knowledge-Guided Expert Aggregation for Large-Scale PDE Pretraining
Oct 2, 2026Dengdi Sun, Xiaoya Zhou, Xiao Wang +3Neural Operators Mixture-of-Experts Models
School of Artificial Intelligence, Anhui University, Hefei 230601, China · School of Computer Science and Technology, Anhui University, Hefei 230601, China
Dynamic Expert Pruning for Multi-Agent Systems
Oct 2, 2026Jabin Koo, Soheil Abbasloo, Sungjae Lee +1Mixture-of-Experts Pruning Mixture-of-Experts Language Models
Pohang University of Science and Technology (POSTECH) · Microsoft Research Asia
Frequency Is Not Sensitivity Identifying Safety-Sensitive Experts in Sparse MoE LLM
Oct 2, 2026Md Nurul Absar Siddiky, Liuwan Zhu, Yingfei DongLLM Safety Mixture-of-Experts Language Models
Department of Electrical and Computer Engineering University of Hawaii at Manoa, Honolulu, HI 96822, USA
Redundancy Meets Synergy: Dependency-aware Expert Selection for MoE via Submodular Optimization
Sep 30, 2026Zheng Lin, Shaoke Fang, Yuxin Zhang +6Submodular Optimization Mixture-of-Experts Pruning
Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg, Luxembourg · Department of Computer Science, Peking University, China · Institute of Space Internet, Fudan University, China +4
MoRA: MoE Pruning via Router Bias Learning and Expert Approximation
Sep 30, 2026Yushuai Sun, Zikun Zhou, Lin Gao +2Mixture-of-Experts Pruning LLM Compression
Harbin Institute of Technology, Shenzhen
DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inference
Sep 30, 2026Xudong Tan, Peng Ye, Ming Xie +4Mixture-of-Experts Pruning Efficient Multimodal Inference
Fudan University · The Chinese University of Hong Kong
DIET: Deletion-response Expert Trimming for Video Diffusion Transformers
Sep 29, 2026Jiachang Zhang, Teng Hu, Bohao Feng +4Mixture-of-Experts Pruning Video Diffusion Models
Xi’an Jiao Tong University, China · Shanghai Jiao Tong University, China · Alibaba Token Hub, Alibaba Group, China +1
BASE: Batch-Aware Selection of Experts Using Predicted Removal Error for Efficient MoE Decoding
Sep 28, 2026Ali Abbasi, Justin Shi, Soheil KolouriMixture-of-Experts Inference LLM Inference Acceleration
College of Connected Computing, Vanderbilt University
SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving
Sep 28, 2026Gunho Park, Kyoungho Jeun, Juntaek Oh +3Mixture-of-Experts Pruning Disaggregated LLM Serving
a2sys · KAIST
EAT: Expert Account Tracker for Efficient MoE Inference
Sep 27, 2026Yuexian Li, Yifei Yang, Zouying Cao +1Mixture-of-Experts Pruning Mixture-of-Experts Language Models
Paris Elite Institute of Technology, Shanghai Jiao Tong University · AGI Institute, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University +1
RAZOR: Pruning Replaceable Experts in LLMs
Sep 24, 2026Mingyang Song, Mao ZhengMixture-of-Experts Pruning LLM Pruning
Foundation Model Department, Tencent, China
You Only Need 2/3 of the Chosen Experts: An Empirical Study of Dynamic Expert Pruning in Fine-Grained MoE LLMs
Sep 22, 2026Yuanteng Chen, Qiwei Lai, Chen Tianqi +7Mixture-of-Experts Pruning Mixture-of-Experts Language Models
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy +1
Higher-order pruning of experts in mixture-of-experts language models
Sep 16, 2026Alex M. Tseng, Prannay Kaul, Luca Zancato +2Mixture-of-Experts Pruning Mixture-of-Experts Language Models
AI Fundamental Research AWS Agentic AI
ESTS at WMT26: Routing-Informed Expert Pruning for Model Compression
Sep 11, 2026Liu O. Martin, Lucas Bandarkar, Nanyun PengModel Compression Mixture-of-Experts Pruning
University of California, Los Angeles
OMP-MoE: Efficient Expert Pruning for Mixture-of-Experts LLMs via Orthogonal Matching Pursuit
Sep 9, 2026Dezhi Li, Lujun Li, Qiyuan Zhu +4Mixture-of-Experts Pruning Mixture-of-Experts Language Models
The Hong Kong University of Science and Technology
Beyond Retraining-Free MoE Compression: A Cost-Normalized Study of Post-Compression Adjustment
Sep 5, 2026Sieun Hyeon, Jaeyoung DoMixture-of-Experts Pruning Fine-Tuning
Department of Electrical and Computer Engineering, Seoul National University · Interdisciplinary Program in Artificial Intelligence, Seoul National University
Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models
Sep 4, 2026Xing Chen, Hengshuai YaoMixture-of-Experts Language Models Mixture-of-Experts Inference
When Load-Balancing Goes Too Far: Expert Pruning in Over-Dispersed Mixture-of-Experts Models
Sep 3, 2026Berkcan Kapusuzoglu, Connor Pryor, Sangwoo Cho +4Mixture-of-Experts Pruning Expert Routing
Capital One, AI Foundations, McLean, VA 22102, USA
PCoMoE: Shifting MoE Inference from Monolithic Expert Selection to Fine-Grained Path Composition
Sep 1, 2026Ziyan Gan, Fangxin Liu, Chenyang Guan +10Mixture-of-Experts Pruning Mixture-of-Experts Inference
1Shanghai Jiao Tong University, China · 2Alibaba Group, China
Benchmarking Composable Compression Techniques in Mixture-of-Experts LLMs
Aug 22, 2026Afsara Benazir, Chen Chen, Rongxiao Qu +3Mixture-of-Experts Inference LLM Evaluation
University of Virginia · Sony AI
MAPLE: MoE Adaptive Plug-and-play Layer-wise Expert allocation
Aug 15, 2026Lie Li, Wen Li, Junxiao Shen +1Mixture-of-Experts Models Mixture-of-Experts Inference
University of Bristol

Oct 27, 2026J/K move · Enter open · S save

CRISP: Co-designing Pruning and Scheduling for Efficient MoE Inference on Edge Servers

Congwei Zhang, Youming Tao, Yifei Zou, Yu Yang+5

School of Electrical Engineering and Computer Science, TU Berlin, Berlin, Germany · School of Computer Science and Technology, Shandong University, Qingdao, China · Department of Data Science, City University of Hong Kong, Hong Kong, China

Mixture-of-Experts Inference Memory-Efficient Inference Mixture-of-Experts Pruning Edge Computing

Open paper AskPDF arXiv

Mixture-of-Experts (MoE) inference on memory-constrained edge platforms is often dominated by expert-weight movement rather than computation. Although each token activates only a small subset of experts, routing is input-dependent, so all experts must remain accessible and memory movement dominates latency. Existing approaches such as uniform pruning and expert dropping overlook two properties of MoE execution: expert combinations are highly skewed in frequency, and experts contribute asymmetrically within each combination. We present CRISP, a combination-aware framework that co-designs pruning and runtime scheduling for efficient MoE serving on memory-constrained edge platforms. CRISP reorders neurons by importance so one stored expert can support multiple execution widths, assigns differentiated execution to hot, warm, and cold combinations based on frequency and expert role, and exploits heterogeneous widths to overlap expert loading with computation. Across Mixtral-8x7B and DeepSeek-MoE-16B, CRISP improves throughput by up to

2.1\times

at the same memory budget while limiting accuracy loss to 3.6% relative to full-width inference. Our results highlight that efficient edge MoE serving requires jointly optimizing which expert capacity is preserved and how expert execution is scheduled.