cs.CLNov 17, 2026

MoE--LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study

Authors: Nikit Srivastava, Joel Dag, René Speck, Axel-Cyrille Ngonga Ngomo

Organizations: Data Science Group, Heinz Nixdorf Institute, Paderborn University, Germany

Abstract

Routed low-rank adapters offer a parameter-efficient means to improve the performance of large language models (LLMs) across tasks. Hence, it is assumed that they can also be used to improve the support of low-resource languages in LLMs, which currently serve English far better than the thousands of other languages spoken worldwide. However, this assumption has never been verified empirically under a shared multilingual evaluation protocol. We address exactly this gap by comparing eight low-rank adaptation (LoRA) methods on a frozen base model using 200 unique languages in more than 9{,}000 evaluation settings. Our results suggest that a reduced-scale search can identify viable configurations of LoRA methods but does not reliably predict their relative performance, as the top-ranking configuration is never best at scale. LoRA reliably improves language-model fit on translation data, consistently degrades reading comprehension, and benefits low-resource languages most. These findings suggest that LoRA methods should be regarded as targeted tools rather than general-purpose means for improving the multilingual capabilities of LLMs.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation

    May 8, 2026Haozhan Tang, Xiuqi Zhu, Xinyin Zhang +3LLM Fine-TuningLow-Rank Adaptation

  2. Which LoRA? An Empirical Study on the Effectiveness of LoRA Techniques During Multilingual Instruction Tuning

    Jun 9, 2026Thamali Wijewardhana, Napoleon H. Reyes, Surangika RanathungaLow-Rank AdaptationCross-Lingual Transfer

  3. Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion

    Apr 20, 2026Chen Zhang, Jiuheng Lin, Zhiyuan Liao +1Language Model EnsemblesLow-Resource Language Processing