Mixture-of-Experts Pruning

Overview

Mixture-of-Experts pruning studies how to shrink models made of specialized subnetworks while retaining their accuracy and efficiency advantages. Methods rank experts using signals such as routing frequency, activation strength, weight or gradient sensitivity, and estimated deletion error; others account for expert interactions or allocate pruning budgets across layers. Approaches range from training-free expert removal and neuron-level trimming to merging, distillation, and dynamic selection at inference, with evaluation spanning language, reasoning, coding, and domain-specific tasks. A central challenge is that importance estimates and safe pruning levels vary by model, task, and deployment setting, so gains in memory or speed must be weighed against capability loss and changes in generation behavior.

Latest papers

CardsList

68 papers14 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. GEAR2^2: Pruned Experts Live On through Gated Reuse of Activated Experts and a Missing-Mass Residual

    Nov 16, 2026Linliang Chen, Yan Zhong, Wang Kang +3Mixture-of-Experts PruningMixture-of-Experts Language Models

  2. Understanding Mixture-of-Experts Language Models Beyond Expert Boundaries

    Nov 10, 2026Zhenhong Zhou, Mao Zheng, Mingyang Song +3Mixture-of-Experts Language ModelsMixture-of-Experts Expert Specialization

  3. SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving

    Nov 4, 2026Erfan Baghaei Potraghloo, Saeid Shokoufa, Seyedarmin Azizi +4LLM SafetyMixture-of-Experts Language Models

  4. Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation

    Nov 4, 2026Hao Cheng, Zhiyu Yang, Bo Ma +1Mixture-of-Experts InferenceExpert Routing

  5. What We Keep of What We Forget: Replacement Operators and Routing Semantics in Training-Free MoE Expert Pruning

    Oct 30, 2026Ezza Ali, Sehrish Bibi, Momina Moetesum +1Mixture-of-Experts PruningExpert Routing

  6. Experts Exhibit Structured Redundancy: Structure-Preserving Expert Pruning for Mixture-of-Experts Large Language Models

    Oct 30, 2026Tian Wang, Yongding Tao, Yihong Dong +3Mixture-of-Experts PruningMixture-of-Experts Language Models

  7. CRISP: Co-designing Pruning and Scheduling for Efficient MoE Inference on Edge Servers

    Oct 27, 2026Congwei Zhang, Youming Tao, Yifei Zou +6Mixture-of-Experts InferenceMemory-Efficient Inference

  8. GS-MoE: Training-Free MoE Compression via Groupwise Subspace Sharing and Pruning

    Oct 23, 2026Carson Cary, Yukai Jin, Yihao Huang +5Mixture-of-Experts ModelsMixture-of-Experts Pruning

  9. Don't Skip the Baseline: A Controlled Comparison of Dynamic Expert Skipping in MoEs

    Oct 20, 2026Jakob Steimle, Thomas Elsken, Daniel Ehrhardt +3Mixture-of-Experts InferenceMixture-of-Experts Pruning

  10. Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models

    Oct 19, 2026Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin +7Mixture-of-Experts PruningMixture-of-Experts Language Models

  11. Fine-Grained Units, Flat Decisions: Hierarchical Micro-Expert Pruning for Mixture-of-Experts Models

    Oct 19, 2026Zukang Xu, Xinpei Gao, Xing Hu +2Channel PruningMixture-of-Experts Pruning

  12. PW-Pruning: Phase-Aware Expert Selection for Speech-Centric Omni-Modal Models

    Oct 15, 2026Junjie Cao, Fengping Tian, Wenjie Zhou +5Mixture-of-Experts PruningMixture-of-Experts Models

  13. Towards Robust Calibration Dataset for Domain-General Mixture of Experts Pruning

    Oct 11, 2026Luis Frentzen Salim, Chia-Chun Wu, Tran Van Nhiem +2Mixture-of-Experts PruningNeural Network Pruning

  14. SPEAR: A Spectral-Disentangled MoE Neural Operator with Knowledge-Guided Expert Aggregation for Large-Scale PDE Pretraining

    Oct 2, 2026Dengdi Sun, Xiaoya Zhou, Xiao Wang +3Neural OperatorsMixture-of-Experts Models

  15. Dynamic Expert Pruning for Multi-Agent Systems

    Oct 2, 2026Jabin Koo, Soheil Abbasloo, Sungjae Lee +1Mixture-of-Experts PruningMixture-of-Experts Language Models

  16. Frequency Is Not Sensitivity Identifying Safety-Sensitive Experts in Sparse MoE LLM

    Oct 2, 2026Md Nurul Absar Siddiky, Liuwan Zhu, Yingfei DongLLM SafetyMixture-of-Experts Language Models

  17. Redundancy Meets Synergy: Dependency-aware Expert Selection for MoE via Submodular Optimization

    Sep 30, 2026Zheng Lin, Shaoke Fang, Yuxin Zhang +6Submodular OptimizationMixture-of-Experts Pruning

  18. MoRA: MoE Pruning via Router Bias Learning and Expert Approximation

    Sep 30, 2026Yushuai Sun, Zikun Zhou, Lin Gao +2Mixture-of-Experts PruningLLM Compression

  19. DIET: Deletion-response Expert Trimming for Video Diffusion Transformers

    Sep 29, 2026Jiachang Zhang, Teng Hu, Bohao Feng +4Mixture-of-Experts PruningVideo Diffusion Models

  20. BASE: Batch-Aware Selection of Experts Using Predicted Removal Error for Efficient MoE Decoding

    Sep 28, 2026Ali Abbasi, Justin Shi, Soheil KolouriMixture-of-Experts InferenceLLM Inference Acceleration

  21. EAT: Expert Account Tracker for Efficient MoE Inference

    Sep 27, 2026Yuexian Li, Yifei Yang, Zouying Cao +1Mixture-of-Experts PruningMixture-of-Experts Language Models

  22. RAZOR: Pruning Replaceable Experts in LLMs

    Sep 24, 2026Mingyang Song, Mao ZhengMixture-of-Experts PruningLLM Pruning

  23. You Only Need 2/3 of the Chosen Experts: An Empirical Study of Dynamic Expert Pruning in Fine-Grained MoE LLMs

    Sep 22, 2026Yuanteng Chen, Qiwei Lai, Chen Tianqi +7Mixture-of-Experts PruningMixture-of-Experts Language Models