Multilingual Language Models

Overview

Multilingual language models study how systems can understand, reason, translate, and generate text across languages, including those with limited training data or distinct scripts. Common approaches include multilingual pretraining and data balancing, continued pretraining and instruction tuning, language-aware tokenization, and transfer methods such as shared representations, adapters, and mixture-of-experts models. Researchers evaluate performance with translation, comprehension, reasoning, and safety benchmarks, while probing internal representations and using inference-time steering to improve language control. A central tension is how to expand language coverage and transfer shared knowledge without degrading performance, reinforcing English or high-resource-language biases, or overlooking differences between languages and scripts.

Latest papers

CardsList

278 papers19 papers in the last four weeks, down 44% on the four weeks before. 0.1% of all new papers.

  1. Language Exposure Matters: Distribution and Order in Multilingual BabyLM Pre-training

    Oct 14, 2026JiWoo Hwang, Lukas Edman, Alexander FraserLanguage Model PretrainingMultilingual Language Models

  2. Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text

    Oct 13, 2026Lukas Edman, Alexander FraserLanguage ModelingLanguage Model Pretraining

  3. Probing the Geometry of Color Representations in Multilingual Language Models

    Oct 13, 2026Pablo Loyola, Edison Marrese-Taylor, Andrés Hoyos-IdroboRepresentation Geometry in Language ModelsMultilingual Language Models

  4. Cross-Lingual Transfer of Evil Vectors Is Directionally Asymmetric

    Oct 13, 2026Juann Kim, Minku KangCross-Lingual TransferActivation Steering

  5. The Multilingual Artificial Hivemind: Do All LLMs Think Alike in Every Language?

    Oct 12, 2026Xaver Raphael Davey, Shubhra Mishra, Anton Adelöw +1Output Diversity in Language ModelsHomogenization in Language Models

  6. HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision

    Oct 12, 2026Iman Barati, Mohammadmostafa Rostamkhani, Mohammad BadzohrehMachine TranslationMultilingual Language Models

  7. Latent Core Tokenizer: Compress, but Meaningfully

    Oct 8, 2026Felermino D. M. A. Ali, Millicent Ochieng, Ogbemi Ekwejunor-Etchie +3Morphology-Aware TokenizationTokenization in Language Models

  8. Language-Specific Effects of Tokenizer Choice in Multilingual Language Models

    Oct 8, 2026Clara Meister, Gül Sena Altıntaş, Antoine BosselutMultilingual Language ModelsMultilingual Language Model Evaluation

  9. Does Modern Standard Arabic (MSA) Dominate Arabic Dialects in LLMs? A Representation-Level Analysis

    Oct 8, 2026Abdu Sallouh, Nicholas Popovič, Michael FärberArabic Language ModelsArabic NLP

  10. Are Language Models Script-Aware?

    Oct 6, 2026David Kletz, Sandra Mitrović, Ljiljana Dolamić +1Multilingual Language ModelsMultilingual Language Model Evaluation

  11. A theory of platonic representations in language models

    Oct 5, 2026Darshil Doshi, Wenjie Zhou, Corinna Elena Wegner +3Belief PropagationMultilingual Language Models

  12. Cross-Lingual Transferability of Training Data Extraction Attacks to Recover Memorized PII

    Oct 5, 2026Alexandru Nazare, Agnese Profico, Nicolò Vania +6Multilingual Language ModelsCross-Lingual Representation Alignment

  13. Zero-shot Dependency Parsing with Unsupervised Cross-Lingual Bootstrapping

    Sep 29, 2026Lalita Lowphansirikul, Attapol Rutherford, Jian Gang Ngui +2Multilingual Language ModelsLow-Resource Language Processing

  14. Cross-Linguistic Effects in Bilingual Phoneme BabyLMs

    Sep 29, 2026Nikitas Theodoropoulos, Maria Lymperaiou, Giorgos FilandrianosMultilingual Language ModelsChild Language Acquisition

  15. Multilinguality in Hybrid Attention LLMs

    Sep 28, 2026Lucas Bandarkar, Junlin Hu, Chenyuan Yang +2Multilingual Language ModelsCross-Lingual Representation Alignment

  16. Feeding BabyLMs Macaroni: Code-Switching Curricula Cause Cross-Lingual Convergence

    Sep 24, 2026Dries Rooryck, Alex Cai, Yonatan Belinkov +2Multilingual Language ModelsLanguage Model Pretraining

  17. Script Choice in LLMs: Evidence for Late-Layer Commitment

    Sep 23, 2026David Kletz, Sandra Mitrović, Itay Sabato +2Multilingual Language ModelsText Generation

  18. Beetle: A Bilingual Model Suite for Modelling Second-Language Processing

    Sep 18, 2026Suchir Salhan, Catherine Arnett, James Michaelov +1Multilingual Language ModelsLanguage Model Pretraining

  19. Beyond Atomic Tokens: Factorizing Syllables for Language Model Pretraining

    Sep 18, 2026Nghia Hieu Nguyen, Thai Bao Huynh, Binh-An Dinh-Le +4TokenizationLanguage Model Pretraining

  20. Why Pretraining Fails to Share Cross-Lingual Knowledge

    Sep 16, 2026Adam Gaber, Uriel Dolev, Elisabeth Fittschen +3Language Model PretrainingMultilingual Language Models

  21. SEA-LION-v4.8: A Technical Report

    Sep 16, 2026Adila Aulia, Ahmed Dabeer, Ahn Jeongmi +38Multilingual Language ModelsLLM Fine-Tuning

  22. Target-Language Generation in Multilingual Models: Activation Steering and Optimal Control

    Sep 15, 2026James A. Michaelov, Carmen Amo Alonso, Tyler A. Chang +1Multilingual Language ModelsMultilingual Language Model Evaluation

  23. To Each Language Its Tokenizer: Modular Tokenizers for Efficient Multilingual LLMs

    Sep 14, 2026Franck Signe, Hippolyte Pilchen, François Yvon +1Multilingual Language ModelsLLM Compression