Multilingual Language Models

Follow this topic Ask about this topic

Multilingual language models study how systems can understand, reason, translate, and generate text across languages, including those with limited training data or distinct scripts. Common approaches include multilingual pretraining and data balancing, continued pretraining and instruction tuning, language-aware tokenization, and transfer methods such as shared representations, adapters, and mixture-of-experts models. Researchers evaluate performance with translation, comprehension, reasoning, and safety benchmarks, while probing internal representations and using inference-time steering to improve language control. A central tension is how to expand language coverage and transfer shared knowledge without degrading performance, reinforcing English or high-resource-language biases, or overlooking differences between languages and scripts.

Latest papers

266 papers39 papers in the last four weeks, up 95% on the four weeks before. 0.2% of all new papers.

LLM Parameters for Math Across Languages: Shared or Separate?
Jun 16, 2026Behzad Shomali, Luisa Victor, Tim Selbach +5Multilingual Language Models Cross-Lingual Reasoning in Language Models
Lamarr Institute · University of Bonn · Fraunhofer IAIS
Not Truly Multilingual: Script Consistency as a Missing Dimension in VLM Evaluation
Jun 15, 2026Prabhjot Singh, Bhushan Pawar, Madhu Reddiboina +1VLM Evaluation Multilingual Language Models
RediMinds Inc., USA · The University of Texas at Austin, USA · Indian AI Research Organization (IAIRO), India
AdaMame: A Training Recipe for Adaptive Multilingual Reasoning
Jun 13, 2026Dayeon Ki, Kevin Duh, Marine CarpuatMultilingual Language Models Cross-Lingual Reasoning in Language Models
University of Maryland · Johns Hopkins University
Equity with Efficiency: An Empirical Study of Tokenizers for Multilingual Large Language Models
Jun 13, 2026Kieron Seven Jun Wei Lee, Muhammad Reza Qorib, Andrew Ivan Soegeng +1Multilingual Language Models Social Bias in Language Models
National University of Singapore · Carnegie Mellon University · SAP
When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs
Jun 12, 2026Boris Marinov, Angira Sharma, Christian Schroeder de Witt +3Multilingual Language Models LLM Interpretability
1UCL · University of Oxford
ArogyaSutra: A Multi-Agent Framework for Multimodal Medical Reasoning in Indic Languages
Jun 11, 2026Tanmoy Kanti Halder, Akash Ghosh, Subhadip Baidya +2Multilingual Language Models RL for Language Model Reasoning
1Indian Institute of Technology Patna · 3Prasannadeb Women’s College · 2Indian Institute of Technology Kanpur
PolyAlign: Conditional Human-Distribution Alignment
Jun 11, 2026L. D. M. S. Sai Teja, Ufaq Khan, Sathira Silva +2Multilingual Language Models LLM Alignment
NIT Silchar, India · MBZUAI, Abu Dhabi, UAE
Language-Aware Token Boosting: LLM Language Confusion Reduction Without Tuning
Jun 8, 2026Trapoom Ukarapol, Pakhapoom Sarapat, Nut ChukamphaengMultilingual Language Models LLM Alignment
§SCB DataX · ‡Tsinghua University · †SCBX
Multilingual Fact-Checking at Scale: Fine-Tuned Compact Models vs LLMs
Jun 7, 2026Pratuat Amatya, Vinay SettyMultilingual Language Models LLM Evaluation
Factiverse AS and University of Stavanger Stavanger, Norway
Sycophancy as a Multilingual Alignment Failure: How Safety Degrades Across Languages, Topics, and Models
Jun 7, 2026Arya Shah, Himanshu Beniwal, Mayank Singh +1Multilingual Language Models Multilingual Language Model Evaluation
IIT Gandhinagar · Asian Institute of Technology
mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages?
Jun 5, 2026Yerzhan Sapenov, Jaromir SavelkaMultilingual Language Models Multilingual Language Model Evaluation
School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA
English-to-Prakrit Machine Translation via Multilingual Transfer Learning
Jun 4, 2026Om Choksi, Smit Kareliya, Shrikant Malviya +1Multilingual Language Models Low-Resource MT
Sardar Vallabhbhai National Institute of Technology, Surat
Multilingual Fine-Tuning via Localized Gradient Conflict Resolution
Jun 4, 2026Long P. Hoang, Yiran Zhao, Wei Lu +1Multilingual Language Models LLM Fine-Tuning
Singapore University of Technology and Design · Salesforce AI Research · Nanyang Technological University
DuDi: Dual-Signal Distillation with Cross-Lingual Verbalizer
Jun 3, 2026Patomporn Payoungkhamdee, Tinnakit Udsa, Jian Gang Ngui +3Multilingual Language Models Small Language Models
School of Information Science and Technology, VISTEC · AI Singapore · MBZUAI
Learning When to Translate for Multilingual Reasoning
Jun 1, 2026Deokhyung Kang, Hyounghun KimMultilingual Language Models Cross-Lingual Reasoning in Language Models
Graduate School of Artificial Intelligence · Department of Computer Science & Engineering
Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource Languages
Jun 1, 2026Saeed Almheiri, Bilal Elbouardi, Salsabila Zahirah Pranida +16Figurative Language Understanding Multilingual Language Models
1Mohamed bin Zayed University of Artificial Intelligence · University of Hamburg · 3Manipal University +4
Multilinguality of Large Language Models From a Structural Perspective
Jun 1, 2026Haruki Sakajo, Yusuke Sakai, Hidetaka Kamigaito +1Multilingual Language Models Representational Similarity Analysis
Nara Institute of Science and Technology (NAIST)
Cross-lingual Self-Consistency for Multilingual Reasoning with Language Models
May 31, 2026Ahmed Elhady, Eneko Agirre, Mikel ArtetxeMultilingual Language Models Cross-Lingual Reasoning in Language Models
HiTZ Center, University of the Basque Country (UPV/EHU) · Reka AI
Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models
May 29, 2026Sanchit Ahuja, Terra BlevinsMultilingual Language Models Continual Learning for LLMs
Northeastern University
Reliable Multilingual Orthopedic Decision Support from Clinical Narratives: Language-Aware Adaptation and Verification-Guided Deferral
May 29, 2026Danish Ali, Li Xiaojian, Sundas Iqbal +1Multilingual Language Models Selective Prediction
School of Computer Science, Wuhan University, China · School of Software, Nanjing University of Information Science and Technology (NUIST), China · Department of Orthopedic, Bahawal Victoria Hospital, Pakistan
The Latin Substrate: How Language Models Represent and Mediate Script Choice
May 29, 2026Daniil Gurgurov, Alan Saji, Katharina Trinley +2Multilingual Language Models LLM Interpretability
D German Research Center for AI (DFKI) · I Indian Institute of Technology, Madras · H University of Hamburg +1
Shared Doubt: Zero-Shot Cross-Lingual Confidence Estimation for Language Models
May 29, 2026Athina Kyriakou, Dennis Ulmer, Ivan TitovMultilingual Language Models Zero-Shot Learning
° ILCC, University of Edinburgh · M ILLC, University of Amsterdam
Speculative Decoding Across Languages
May 28, 2026Nirajan Paudel, Michael Ginn, Luc De Nardi +1Multilingual Language Models Speculative Decoding
University of Colorado
Cross-Lingual Steering for Figurative Language Generation
May 28, 2026Linfeng Liu, Tiffany Zhan, Louie Hong Yao +2Multilingual Language Models Language Model Steering
Department of Computer Science, University of Cincinnati · School of Computer Science, Carnegie Mellon University · Independent Researcher
Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation
May 28, 2026M. Ali Bayram, Banu Diri, Savaş YıldırımMultilingual Language Models Sentence Embeddings
Department of Computer Engineering Yıldız Technical University Istanbul, Turkey · Department of Computer Engineering Istanbul Bilgi University Istanbul, Turkey
Leveraging Routing Dynamics in Mixture-of-Experts Models for Efficient Language Adaptation
May 28, 2026Aditi Khandelwal, Marius Mosbach, Verna Dankers +2Multilingual Language Models Continual Learning for LLMs
Mila – Quebec AI Institute & McGill University
Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning
May 28, 2026Shunta Asano, Jeonghun Baek, Toshihiko YamasakiMultilingual Language Models Code-Switching
The University of Tokyo
Disentangling Language Roles in Multilingual LLM Task Execution
May 26, 2026Qishi Zhan, Minxuan Hu, Seoyeon Jang +7Multilingual Language Models Multilingual Language Model Evaluation
Marquette · Cornell · UC San Diego +7
Rethinking the Multilingual Reasoning Gap with Layer Swap
May 26, 2026Maxence Lasbordes, Amélie Chatelain, Djamé SeddahMultilingual Language Models Multilingual Language Model Evaluation
1LightOn, Paris · 2Inria, Paris
CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations
May 25, 2026Mike Zhang, Ali Basirat, Desmond ElliottMultilingual Language Models LLM Alignment
Department of Computer Science (DIKU), University of Copenhagen · Centre for Language Technology (CST), University of Copenhagen · Pioneer Centre for Artificial Intelligence

Jun 16, 2026J/K move · Enter open · S save

LLM Parameters for Math Across Languages: Shared or Separate?

Behzad Shomali, Luisa Victor, Tim Selbach, Ali Hamza Bashir+4

Lamarr Institute · University of Bonn · Fraunhofer IAIS

Multilingual Language Models Cross-Lingual Reasoning in Language Models LLM Interpretability LLM Reasoning Cross-Lingual Transfer

Open paper AskPDF arXiv

Large language models (LLMs) exhibit substantial cross-lingual variation in mathematical reasoning performance, but it remains unclear whether these differences reflect language-specific parameters or a shared mechanism that manifests differently by language. We present a cross-lingual mechanistic analysis of mathematical reasoning in LLMs, enabling us to localize and compare model parameters that support mathematical reasoning across languages. We find that the extracted math-associated parameters exhibit partial cross-lingual overlap, with the strongest overlap concentrated in intermediate model layers. We further observe that English consistently produces the largest set of math-relevant parameters, whereas lower-resource languages reveal smaller sets of relevant parameters. These results suggest that math-related behavior in multilingual LLMs is neither fully language-invariant nor fully language-specific, but instead exhibits partial cross-lingual parameter overlap with systematic language-dependent differences.