cs.AINov 16, 2026

GEAR2^2: Pruned Experts Live On through Gated Reuse of Activated Experts and a Missing-Mass Residual

Authors: Linliang Chen, Yan Zhong, Wang Kang, Sai Li, Ji Wu, Ji Liu

Organizations: 1Hangzhou International Innovation Institute, Beihang University, Hangzhou, China · School of Mathematical Sciences, Peking University, Beijing, China · School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China · Department of Electronic Engineering, Tsinghua University, Beijing, China

Abstract

Mixture-of-Experts (MoE) Large Language Models (LLMs) activate only a few experts per token while generally keeping all the experts in GPU memory for high efficiency. Expert pruning reduces this GPU memory footprint cost, yet discards the contribution of pruned experts. Existing remedies either rely on static substitutes, merge experts at the cost of altering their functions, or require costly recovery training. In this paper, we find that the functionalities of pruned experts can be efficiently reconstructed from the activated remaining experts. We propose GEAR (Gated Expert Activation Reuse), which reuses the scaled outputs of reliable activated experts without any training or additional expert computation. In addition, we propose GEAR2^2 to further recover what reuse cannot express with an ultra-lightweight residual. On five MoE LLMs from 16B to 106B parameters with 25% and 50% of experts pruned, GEAR improves over its training-free counterparts in nearly every setting (up to 4 pp.) in terms of accuracy, and GEAR2^2 outperforms baseline approaches (up to 2.75 pp.) in every setting on nine zero-shot benchmarks, with larger advantages (up to 2.72 pp. higher) under more aggressive pruning (up to 50%).

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. REAM: Merging Improves Pruning of Experts in LLMs

    Apr 6, 2026Saurav Jha, Maryam Hashemzadeh, Ali Saheb Pasand +3LLM CompressionMixture-of-Experts Pruning

  2. Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models

    Oct 19, 2026Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin +7Mixture-of-Experts PruningMixture-of-Experts Language Models