cs.CLNov 4, 2026

Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation

Authors: Hao Cheng, Zhiyu Yang, Bo Ma, Hanzhang Jia

Organizations: Zhejiang Gongshang University · Chang'an University · University of Sussex

Abstract

Expert deactivation at inference time changes both token routing and expert weights, and its effect on performance depends on how these changes are implemented. We compare removal, freezing, remaining-expert renormalization, and unnormalized re-routing on identical disabled sets across five instruction-tuned checkpoints from four model families, with a base OLMoE checkpoint as an additional comparison. We develop a weight-sum model that relates the loss of expert contributions and changes in branch scale to the weight rule used during training. With half of the experts deactivated according to activation-based rankings, remaining-expert renormalization yields the highest perplexity in models trained with raw router probabilities, whereas unnormalized re-routing yields the highest perplexity in models trained with normalized weights. Perplexity ranges from 1.4 to 73 times the intact baseline when the training rule is retained, and from 14 to 28,740 times the baseline under a rule mismatch. Task outcomes vary across evaluation types: on OLMoE, GSM8K strict-match accuracy falls to zero under all four implementations. Extending the model to changes in the number of active experts yields a further prediction, supported by the OLMoE experiments: the perplexity penalty from additional renormalization increases as fewer experts are activated.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. What We Keep of What We Forget: Replacement Operators and Routing Semantics in Training-Free MoE Expert Pruning

    Oct 30, 2026Ezza Ali, Sehrish Bibi, Momina Moetesum +1Mixture-of-Experts PruningExpert Routing

  2. Fewer Expert Transfers Without Changing Routing: Expert Eviction for MoE Offloading

    Nov 11, 2026Pierre-Luc Bacon, Diego Calanzone, Jean-Maxime LaroucheMixture-of-Experts InferenceLLM Inference Acceleration