Multilingual Language Models

Ask about this topic

Overview

Multilingual language models study how systems can understand, reason, translate, and generate text across languages, including those with limited training data or distinct scripts. Common approaches include multilingual pretraining and data balancing, continued pretraining and instruction tuning, language-aware tokenization, and transfer methods such as shared representations, adapters, and mixture-of-experts models. Researchers evaluate performance with translation, comprehension, reasoning, and safety benchmarks, while probing internal representations and using inference-time steering to improve language control. A central tension is how to expand language coverage and transfer shared knowledge without degrading performance, reinforcing English or high-resource-language biases, or overlooking differences between languages and scripts.

Latest papers

CardsList

266 papers39 papers in the last four weeks, up 95% on the four weeks before. 0.2% of all new papers.

  1. Right at My Level: A Unified Multilingual Framework for Proficiency-Aware Text Simplification

    Apr 7, 2026Jinhong Jeong, Junghun Park, Youngjae YuSecond Language AcquisitionText Simplification

    Yonsei University · Seoul National University

  2. Multilingual Language Models Encode Script Over Linguistic Structure

    Apr 6, 2026Aastha A K Verma, Anwoy Chatterjee, Mehak Gupta +1Multilingual Language ModelsCross-Lingual Representation Alignment

    Indian Institute of Technology Delhi, New Delhi, India · Indian Institute of Technology Delhi, Abu Dhabi, UAE

  3. LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering

    Apr 4, 2026Sing Hieng Wong, Hassan Sajjad, A. B. SiddiqueLanguage Model SteeringMultilingual Language Models

    Department of Computer Science, University of Kentucky, USA · Department of Computer Science, Dalhousie University, Canada

  4. One Model to Translate Them All? A Journey to Mount Doom for Multilingual Model Merging

    Apr 3, 2026Baban Gain, Trilok Nath Singh, Asif EkbalMultilingual Language ModelsLanguage Model Merging

    Indian Institute of Technology Patna, India

  5. Multiple-Debias: A Full-process Debiasing Method for Multilingual Pre-trained Language Models

    Apr 3, 2026Haoyu Liang, Peijian Zeng, Wentao Huang +2Social Bias in Language ModelsMultilingual Language Models

    School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China · School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou, China

  6. Positional Cognitive Specialization: Where Do LLMs Learn To Comprehend and Speak Your Language?

    Apr 1, 2026Luis Frentzen Salim, Lun-Wei Ku, Hsing-Kuo Kenneth PaoMultilingual Language ModelsLLM Fine-Tuning

    Institute of Information Science, Academia Sinica · Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology

  7. Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models

    Mar 31, 2026Linda Zeng, Steven Y. Feng, Michael C. FrankMultilingual Language ModelsMultilingual Language Model Evaluation

    Stanford University

  8. On the limited utility of parallel data for learning shared multilingual representations

    Mar 30, 2026Julius Leino, Jörg TiedemannCross-Lingual Representation AlignmentMultilingual Language Models

    University of Helsinki

  9. Merge and Conquer: Instructing Multilingual Models by Adding Target Language Weights

    Mar 30, 2026Eneko Valero, Maria Ribalta i Albado, Oscar Sainz +2Language Model MergingMultilingual Language Models

    HiTZ Center - Ixa, University of the Basque Country UPV/EHU

  10. AMALIA Technical Report: A Fully Open Source Large Language Model for European Portuguese

    Mar 27, 2026Afonso Simplício, Gonçalo Vinagre, Miguel Moura Ramos +19Portuguese Language ModelsLLM Training

    NOVA School of Science and Technology · NOVA LINCS · Instituto de Telecomunicações +2

  11. A Universal Vibe? Finding and Controlling Language-Agnostic Informal Register with SAEs

    Mar 27, 2026Uri Z. Kialy, Avi Shtarkberg, Ayal KleinMultilingual Language ModelsMechanistic Interpretability

    Ariel University

  12. Optimizing Multilingual LLMs via Federated Learning: A Study of Client Language Composition

    Mar 25, 2026Aleix Sant, Jordi Luque, Carlos EscolanoFederated LearningFederated Fine-Tuning

    Scientific Research, Telefónica Innovación Digital · Universitat Politècnica de Catalunya

  13. MedAidDialog: A Multilingual Multi-Turn Medical Dialogue Dataset for Accessible Healthcare

    Mar 25, 2026Shubham Kumar Nigam, Suparnojit Sarkar, Piyush PatelMedical LLMsText Generation

    University of Birmingham, Dubai, United Arab Emirates · Heritage Institute of Technology, Kolkata, India · Madan Mohan Malaviya University of Technology, India

  14. SAGE Celer 2.6 Technical Card

    Mar 24, 2026SAGEA Research Team, Basab Jha, Firoj Paudel +8Language Model Self-VerificationMultilingual Language Models

    SAGEA · Tribhuwan University | Vedas College · Tribhuwan University | Madan Bhandari Memorial College +2

  15. LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation

    Mar 23, 2026Hailay Teklehaymanot, Dren Fazlija, Wolfgang NejdlLow-Resource Language ProcessingWord Embeddings

    L3S Research Center, Hannover, Germany

  16. MzansiText and MzansiLM: An Open Corpus and Decoder-Only Language Model for South African Languages

    Mar 21, 2026Anri Lombard, Simbarashe Mawere, Temi Aina +5African Language ProcessingMultilingual Language Models

    University of Cape Town, Cape Town, South Africa

  17. Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders

    Mar 20, 2026Debajyoti Mazumder, Divyansh Pathak, Prashant Kodali +1Multilingual Language ModelsCode-Switching

    Indian Institute of Science Education and Research, Bhopal, India · Microsoft Corporation

  18. VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models

    Mar 19, 2026Chonghan Liu, Yimin Du, Qi An +8RL for Language ModelsLow-Resource MT

    1Qiyuan Tech.

  19. Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders

    Mar 19, 2026Yana Veitsman, Yihong Liu, Hinrich SchützeCross-Lingual Representation AlignmentCross-Lingual Transfer

    University of Göttingen · LMU Munich · Munich Center for Machine Learning (MCML)

  20. ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws

    Mar 18, 2026Xuyang Cao, Qianying Liu, Chuan Xiao +7Language Model Scaling LawsMultilingual Language Models

    NII LLMC · The University of Osaka · Nara Institute of Science and Technology +2

  21. Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality

    Mar 18, 2026Mengyu Bu, Yang FengMultilingual Language ModelsCross-Lingual Transfer

    Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China

  22. Multilingual TinyStories: A Synthetic Combinatorial Corpus of Indic Children's Stories for Training Small Language Models

    Mar 15, 2026Deepon Halder, Angira MukherjeeSynthetic Data Generation for Language ModelsLow-Resource Language Processing

    AI4Bharat, IIEST Shibpur · IIEST Shibpur

  23. OasisSimp: An Open-source Asian-English Sentence Simplification Dataset

    Mar 14, 2026Hannah Liu, Muxin Tian, Iqra Ali +8Text SimplificationLow-Resource Language Processing

    University of Toronto · Queen Mary University of London · Yale University +5

  24. Cross-Lingual Transfer and Parameter-Efficient Adaptation in the Turkic Language Family: A Theoretical Framework for Low-Resource Language Models

    Mar 13, 2026O. Ibrahimzade, K. TabasaranskyCross-Lingual TransferMultilingual Language Models

  25. Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation

    Mar 13, 2026Yifeng Liu, Siqi Ouyang, Yatish Hosmane Revanasiddappa +1Low-Resource MTMultilingual Language Models

    Carnegie Mellon University

  26. Learning from Child-Directed Speech in Two-Language Scenarios: A French-English Case Study

    Mar 13, 2026Liel Binyamin, Elior SulemLanguage Model PretrainingLanguage Modeling

    Faculty of Computer and Information Science, Institute for Applied AI Research Data Science Research Center Ben-Gurion University of the Negev

  27. Tiny Aya: Bridging Scale and Multilingual Depth

    Mar 12, 2026Alejandro R. Salamanca, Diana Abagyan, Daniel D'souza +23Multilingual Language ModelsMachine Translation

    Cohere Labs · Cohere

  28. Multi-lingual Multi-institutional Electronic Health Record based Predictive Model

    Mar 11, 2026Kyunghoon Hur, Heeyoung Kwak, Jinsu Jang +2Electronic Health RecordsCross-Lingual Transfer

    Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology, Daejeon 34141, Republic of Korea · NAVER Digital Healthcare LAB, Republic of Korea · Department of Health and Medical Information, Ansan University, Ansan-si, Gyeonggi-do 15328, Republic of Korea +1

  29. TildeOpen LLM: Leveraging Curriculum Learning to Achieve Equitable Language Representation

    Mar 9, 2026Toms Bergmanis, Martins Kronis, Ingus Jānis Pretkalniņš +5Multilingual Language ModelsOpen-Weight Language Models

    Tilde, Latvia

  30. NeuronMoE: Neuron-Guided Mixture-of-Experts for Efficient Multilingual LLM Extension

    Mar 5, 2026Rongzhi Li, Hitomi YanakaMultilingual Language ModelsMixture-of-Experts Language Models

    The University of Tokyo · Riken · Tohoku University