Multilingual Language Models

Ask about this topic

Overview

Multilingual language models study how systems can understand, reason, translate, and generate text across languages, including those with limited training data or distinct scripts. Common approaches include multilingual pretraining and data balancing, continued pretraining and instruction tuning, language-aware tokenization, and transfer methods such as shared representations, adapters, and mixture-of-experts models. Researchers evaluate performance with translation, comprehension, reasoning, and safety benchmarks, while probing internal representations and using inference-time steering to improve language control. A central tension is how to expand language coverage and transfer shared knowledge without degrading performance, reinforcing English or high-resource-language biases, or overlooking differences between languages and scripts.

Latest papers

CardsList

265 papers39 papers in the last four weeks, up 95% on the four weeks before. 0.2% of all new papers.

  1. Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry

    Feb 27, 2026Kyle Elliott MathewsonMultilingual Language ModelsRepresentation Geometry in Language Models

    University of Alberta

  2. Identifying Crucial Attention Heads for Multilingual Language Models: Retrieval and Retrieval-Transition Heads

    Feb 25, 2026Shaswat Patel, Vishvesh Trivedi, Yue Han +2Multilingual Language ModelsCross-Lingual Reasoning in Language Models

    New York University

  3. Enhancing Multilingual Embeddings via Multi-Way Parallel Text Alignment

    Feb 25, 2026Barah Fazili, Koustava GoswamiCross-Lingual Representation AlignmentMultilingual Language Models

    IIT Bombay, Mumbai, India · Adobe Research, Bangalore, India

  4. MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation

    Feb 24, 2026Daniel Tamayo, Iñaki Lacunza, Paula Rivera-Hidalgo +4Multilingual Language ModelsLanguage Model Pretraining

    Barcelona Supercomputing Center

  5. BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop

    Feb 23, 2026Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul +7Language Model PretrainingSmall Language Models

    IBM Research, MIT · ETH Zürich · Cornell University +7

  6. Multilingual Large Language Models do not comprehend all natural languages to equal degrees

    Feb 23, 2026Natalia Moskvina, Raquel Montero, Masaya Yoshida +8Multilingual Language ModelsMultilingual Language Model Evaluation

    Universitat Autònoma de Barcelona · Institució Catalana de Recerca i Estudis Avançats (ICREA) · Radboud University Nijmegen +1

  7. Aladdin-FTI @ AMIYA Three Wishes for Arabic NLP: Fidelity, Diglossia, and Multidialectal Generation

    Feb 18, 2026Jonathan Mutal, Perla Al Almaoui, Simon Hengchen +1Arabic NLPArabic Dialects

    Faculté de traduction et d’interprétation, Université de Genève

  8. Adaptive Engram Memory System for Indonesian Language Model: Generative AI Based on TOBA LM for Batak and Minang Language

    Feb 17, 2026Hokky Situngkir, Kevin Siringoringo, Andhika Bernard LumbantobingLanguage Model PretrainingMultilingual Language Models

    AI Research Center IT Del & Bandung Fe Institute · Bandung Fe Institute · Bandung Fe Institute & Adjunct Science Fellow in InaAI

  9. ÜberWeb: Insights from Multilingual Curation for a 20-Trillion-Token Dataset

    Feb 16, 2026DatologyAI, :, Aldo Gael Carranza +32Multilingual Language ModelsTraining Data Curation

    DatologyAI

  10. Crowdsourcing Piedmontese to Test LLMs on Non-Standard Orthography

    Feb 16, 2026Gianluca Vico, Jindřich LibovickýTokenization in Language ModelsLow-Resource Language Processing

    Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics, Malostranské náměstí 25, 118 00 Praha, Czech Republic

  11. Language Steering for Multilingual In-Context Learning

    Feb 2, 2026Neeraja Kirtane, Kuan-Hao HuangMultilingual Language ModelsCross-Lingual Transfer

    Texas A&M University

  12. Cross-Lingual Activation Steering for Multilingual Language Models

    Jan 23, 2026Rhitabrat Pokharel, Ameeta Agrawal, Tanay NagarMultilingual Language ModelsLanguage Model Steering

    Department of Computer Science, Portland State University, USA · Independent Researcher

  13. HOMURA: Taming the Sand-Glass for Time-Constrained LLM Translation via Reinforcement Learning

    Jan 15, 2026Ziang Cui, Mengran Yu, Chenyu Shi +2Multilingual Language ModelsRL for Language Models

    Bilibili Inc., Shanghai, China

  14. AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages

    Jan 10, 2026Hao Yu, Tianyi Xu, Michael A. Hedderich +3Language Model PretrainingMultilingual Language Models

    McGill University, Canada · Mila-Quebec AI Institute, Canada · LMU Munich & Munich Center for Machine Learning, Germany +2

  15. What Matters When Building Universal Multilingual Named Entity Recognition Models?

    Jan 9, 2026Jonas Golde, Patrick Haller, Alan AkbikNamed Entity RecognitionMultilingual Language Models

    Humboldt Universität zu Berlin

  16. LMSpell: Spell Correction with Pre-Trained Language Models

    Dec 5, 2025Akesh Gunathilake, Nadil Karunarathna, Tharusha Bandaranayake +3Low-Resource Language ProcessingLanguage Model Post-Training

    Department of Computer Science and Engineering, University of Moratuwa, Katubedda, 10400, Sri Lanka · School of Mathematical and Computational Sciences, Massey University, Auckland, 102904, New Zealand

  17. AyurParam: A State-of-the-Art Bilingual Language Model for Ayurveda

    Nov 4, 2025Sravan Gorugantu, Mohd Nauman, Vijay Devane +7Medical LLMsHealthcare

    BharatGen Team

  18. Language steering in latent space to mitigate unintended code-switching

    Oct 11, 2025Andrey Goncharov, Nikolai Kondusov, Alexey ZaytsevLanguage Model SteeringMultilingual Language Models

  19. Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models

    Oct 3, 2025Ej Zhou, Caiqi Zhang, Tiancheng Hu +4Multilingual Language ModelsMultilingual Language Model Evaluation

    Language Technology Lab, University of Cambridge

  20. SinLlama -- A Large Language Model for Sinhala

    Aug 12, 2025H. W. K. Aravinda, Rashad Sirajudeen, Samith Karunathilake +3Language Model PretrainingMultilingual Language Models

    Dept. of Computer Science & Engineering, University of Moratuwa, Sri Lanka. · School of Mathematical and Computational Sciences, Massey University, Auckland, New Zealand. · Central Scientific Instruments Organisation, Academy of Scientific and Innovative Research, Chandigarh, India. +1

  21. Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages

    Jul 15, 2025Lyzander Marciano Andrylie, Inaya Rahmanisa, Mahardika Krisna Ihsani +3Multilingual Language ModelsRepresentation Learning

    Faculty of Computer Science, Universitas Indonesia · Department of Natural Language Processing, MBZUAI

  22. NMIXX: Domain-Adapted Neural Embeddings for Cross-Lingual eXploration of Finance

    Jul 13, 2025Hanwool Lee, Sara Yu, Yewon Hwang +4Multilingual Language ModelsText Embeddings

    FinancialNLPLab, MODULABS Shinhan Securities Seoul, Republic of Korea · FinancialNLPLab, MODULABS KT Seoul, Republic of Korea · FinancialNLPLab, MODULABS EMRO Seoul, Republic of Korea +4

  23. Language Specific Knowledge: Do Models Know Better in X than in English?

    May 21, 2025Ishika Agarwal, Nimet Beyza Bozdag, Dilek Hakkani-TürMultilingual Language ModelsMultilingual QA

    Department of Computer Science University of Illinois, Urbana-Champaign

  24. In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models

    Mar 3, 2025David Ponce, Thierry EtchegoyhenMultilingual Language ModelsMultilingual Language Model Evaluation

    Vicomtech Foundation, Basque Research and Technology Alliance (BRTA) · University of the Basque Country - EHU

  25. Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian

    May 22, 2024Aleksandr Nikolich, Konstantin Korolev, Sergei Bratchikov +2Language Model PretrainingMultilingual Language Models

    ITMO University · HSE University · Vikhr +2