Multilingual Language Models
Overview
Multilingual language models study how systems can understand, reason, translate, and generate text across languages, including those with limited training data or distinct scripts. Common approaches include multilingual pretraining and data balancing, continued pretraining and instruction tuning, language-aware tokenization, and transfer methods such as shared representations, adapters, and mixture-of-experts models. Researchers evaluate performance with translation, comprehension, reasoning, and safety benchmarks, while probing internal representations and using inference-time steering to improve language control. A central tension is how to expand language coverage and transfer shared knowledge without degrading performance, reinforcing English or high-resource-language biases, or overlooking differences between languages and scripts.
Latest papers
278 papers19 papers in the last four weeks, down 44% on the four weeks before. 0.1% of all new papers.
SinLlama -- A Large Language Model for Sinhala
Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages
NMIXX: Domain-Adapted Neural Embeddings for Cross-Lingual eXploration of Finance
Language Specific Knowledge: Do Models Know Better in X than in English?
In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models
Multilingual Hallucination Gaps in Large Language Models
Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian
Vikhr'' refers to the name of the Mistral LLM series and means a strong gust of wind.'' Unlike previous Russian-language models that typically rely on LoRA adapters on top of English-oriented models, sacrificing performance for lower training costs, Vikhr features an adapted tokenizer vocabulary and undergoes the continued pre-training and instruction tuning of all weights. This not only enhances the model's performance but also significantly improves its computational and contextual efficiency. We also expanded the instruction datasets and corpora for continued pre-training. The model weights, instruction sets, and code are publicly available.