Multilingual Language Models

Overview

Multilingual language models study how systems can understand, reason, translate, and generate text across languages, including those with limited training data or distinct scripts. Common approaches include multilingual pretraining and data balancing, continued pretraining and instruction tuning, language-aware tokenization, and transfer methods such as shared representations, adapters, and mixture-of-experts models. Researchers evaluate performance with translation, comprehension, reasoning, and safety benchmarks, while probing internal representations and using inference-time steering to improve language control. A central tension is how to expand language coverage and transfer shared knowledge without degrading performance, reinforcing English or high-resource-language biases, or overlooking differences between languages and scripts.

Latest papers

CardsList

278 papers19 papers in the last four weeks, down 44% on the four weeks before. 0.1% of all new papers.

  1. SinLlama -- A Large Language Model for Sinhala

    Aug 12, 2025H. W. K. Aravinda, Rashad Sirajudeen, Samith Karunathilake +3Language Model PretrainingMultilingual Language Models

  2. Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages

    Jul 15, 2025Lyzander Marciano Andrylie, Inaya Rahmanisa, Mahardika Krisna Ihsani +3Multilingual Language ModelsRepresentation Learning

  3. NMIXX: Domain-Adapted Neural Embeddings for Cross-Lingual eXploration of Finance

    Jul 13, 2025Hanwool Lee, Sara Yu, Yewon Hwang +4Multilingual Language ModelsText Embeddings

  4. Language Specific Knowledge: Do Models Know Better in X than in English?

    May 21, 2025Ishika Agarwal, Nimet Beyza Bozdag, Dilek Hakkani-TürMultilingual Language ModelsMultilingual QA

  5. In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models

    Mar 3, 2025David Ponce, Thierry EtchegoyhenMultilingual Language ModelsMultilingual Language Model Evaluation

  6. Multilingual Hallucination Gaps in Large Language Models

    Oct 23, 2024Cléa Chataigner, Afaf Taïk, Golnoosh FarnadiHallucination in Language ModelsMultilingual Language Model Evaluation

  7. Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian

    May 22, 2024Aleksandr Nikolich, Konstantin Korolev, Sergei Bratchikov +2Language Model PretrainingMultilingual Language Models