Cross-Lingual Representation Alignment

Follow this topic Ask about this topic

The field studies how words, sentences, and internal model features in different languages correspond, enabling knowledge and task performance to transfer across languages. Common approaches use parallel-text contrastive learning, mapping or optimal-transport methods, and inference-time interventions; researchers also probe whether shared structure emerges without joint training. Applications include multilingual retrieval, reasoning, safety, speech, and multimodal tasks, especially for low-resource languages. A central tension is that greater geometric alignment does not always improve downstream transfer, and models must share meaning without erasing differences in language, script, or culture.

Latest papers

69 papers16 papers in the last four weeks, up 78% on the four weeks before. 0.1% of all new papers.

Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality
Mar 18, 2026Mengyu Bu, Yang FengMultilingual Language Models Cross-Lingual Transfer
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech
Mar 17, 2026Omnilingual SONAR Team, João Maria Janeiro, Pere-Lluís Huguet Cabot +17Sentence Embeddings Cross-Lingual Representation Alignment
FAIR at Meta
Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry
Feb 27, 2026Kyle Elliott MathewsonMultilingual Language Models Representation Geometry in Language Models
University of Alberta
Multilingual Safety Alignment Via Sparse Weight Editing
Feb 26, 2026Jiaming Liang, Zhaoxin Wang, Handing WangLLM Safety Alignment LLM Safety
School of Artificial Intelligence, Xidian University.
Enhancing Multilingual Embeddings via Multi-Way Parallel Text Alignment
Feb 25, 2026Barah Fazili, Koustava GoswamiCross-Lingual Representation Alignment Multilingual Language Models
IIT Bombay, Mumbai, India · Adobe Research, Bangalore, India
Cross-lingual Matryoshka Representation Learning across Speech and Text
Feb 23, 2026Yaya Sy, Dioula Doucouré, Christophe Cerisara +1Representation Learning Audio-Text Retrieval
LORIA, CNRS · Soynade Research
Cross-Lingual Sentiment Misalignment: Auditing Multilingual Language Models for Inversion Risk, Dialectal Representation, and Affective Stability
Feb 19, 2026Nusrat Jahan Lia, Shubhashis Roy DiptaDialect Bias in Language Models Cross-Lingual Consistency in Language Models
Institute of Information Technology, University of Dhaka, Dhaka, Bangladesh · University of Maryland, Baltimore County, Baltimore, Maryland, USA
Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment
Feb 18, 2026Yuyan Bu, Xiaohao Liu, ZhaoXing Ren +2LLM Safety Alignment Cross-Lingual Consistency in Language Models
Beijing Academy of Artificial Intelligence · National University of Singapore · Institute for Artificial Intelligence, Peking University
A quantitative analysis of semantic information in deep representations of text and images
May 21, 2025Santiago Acevedo, Andrea Mascaretti, Riccardo Rende +3Representational Similarity Analysis Cross-Modal Representation Learning
Scuola Internazionale Superiore di Studi Avanzati (SISSA) · Universitat Pompeu Fabra (UPF) · Catalan Institute of Research and Advanced Studies (ICREA) and Universitat Pompeu Fabra (UPF)

Mar 18, 2026J/K move · Enter open · S save

Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality

Mengyu Bu, Yang Feng

Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China

Multilingual Language Models Cross-Lingual Transfer Low-Resource Language Processing Cross-Lingual Representation Alignment

Open paper AskPDF arXiv

Large language models (LLMs) exhibit strong general intelligence, yet their multilingual performance remains highly imbalanced. Although LLMs encode substantial cross-lingual knowledge in a unified semantic space, they often struggle to reliably interface this knowledge with low-resource or unseen languages. Fortunately, pretrained encoder-decoder translation models already possess balanced multilingual capability, suggesting a natural complement to LLMs. In this work, we propose XBridge, a compositional encoder-LLM-decoder architecture that offloads multilingual understanding and generation to external pretrained translation models, while preserving the LLM as an English-centric core for general knowledge processing. To address the resulting representation misalignment across models, we introduce lightweight cross-model mapping layers and an optimal transport-based alignment objective, enabling fine-grained semantic consistency for multilingual generation. Experiments on four LLMs across multilingual understanding, reasoning, summarization, and generation indicate that XBridge outperforms strong baselines, especially on low-resource and previously unseen languages, without retraining the LLM.