cs.CLNov 5, 2026

English Calibration Erodes Lithuanian in Expert-Pruned Mixture-of-Experts Language Models

Authors: Tomas Rasymas, Povilas Kvedaras

Organizations: Hostinger Research, Lithuania

Abstract

Expert pruning compresses Mixture-of-Experts (MoE) language models by removing the experts that a calibration set marks as unimportant. Studies of high-resource languages largely find that English calibration transfers well to other languages. We test this for Lithuanian: we pruned four MoE models with REAP, removing 25% or 50% of the experts with English, Lithuanian or mixed calibration, and evaluated both languages on reading comprehension, translation and language modelling. English calibration does not transfer to Lithuanian: at 25% pruning, English comprehension fell by a median of 1.7 points but Lithuanian by 21.4. The transfer is asymmetric. Lithuanian calibration removed most of the Lithuanian loss and never significantly hurt English comprehension, which was on average 2.5 points better than under English calibration; only English language modeling got slightly worse. Routing suggests why: in the last layers, Lithuanian sends 55-72% of its expert selections to just eight experts, against 19-39% for English, and English calibration removes some of these experts. Compressed models should be calibrated, and evaluated, in every language they must serve.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Towards Robust Calibration Dataset for Domain-General Mixture of Experts Pruning

    Oct 11, 2026Luis Frentzen Salim, Chia-Chun Wu, Tran Van Nhiem +2Mixture-of-Experts PruningNeural Network Pruning

  2. Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts

    May 27, 2026Liu O. Martin, Lucas Bandarkar, Nanyun PengMixture-of-Experts PruningLLM Pruning