Representation Alignment

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 82

All topics

Soft Token Alignment for Cross-Lingual Reasoning
Jun 25, 2026Jiayi He, Jungsoo Park, Wei Xu +1Multilingual Language Models Cross-Lingual Reasoning in Language Models
Georgia Institute of Technology
Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos
Jun 23, 2026Danze Chen, Yanzhe Chen, Qiming Huang +3Visuomotor Policy Learning Vision-Language-Action Models
Show Lab, National University of Singapore
Subspace-Constrained Federated Learning with Low-Rank Adaptation
Jun 21, 2026Neranjan Senarath, Rohit Muralitharan, Sadia AsifFederated Learning Aggregation Non-IID Federated Learning
Department of ECSE Rensselaer Polytechnic Institute Troy, NY 12180, USA · Department of Computer Science Rensselaer Polytechnic Institute Troy, NY 12180, USA
CraBERT: Efficient Phoneme Encoder Pre-Training via Cascade Fusion of Subword Representations for Text-to-Speech
Jun 15, 2026Dong Yang, Yuki Saito, Wataru Nakata +1Language Model Pretraining Representation Learning
The University of Tokyo, Japan
Learning aligned EEG representations with subject-specific encoders
Jun 15, 2026Bruna J. Lopes, Gabriel Schwartz, Sylvain Chevallier +2Cross-Subject EEG Decoding MI Classification
University of São Paulo, São Paulo, Brazil · Université Paris-Saclay, Inria TAU team, LISN-CNRS, France · Institut de neuromodulation, GHU Paris, psychiatrie et neurosciences, centre hospitalier Sainte-Anne, pôle hospitalo-universitaire 15, Université Paris Cité, Paris, France +3
Scalable and Interpretable Representation Alignment with Ordinal Similarity
Jun 15, 2026Diogo Soares, Pankhil Gawade, Andrea Dittadi +1Representational Similarity Analysis Representation Alignment
Institute of AI for Health, Helmholtz Munich · School of Computation, Information and Technology, Technical University of Munich · Faculty of Mathematics, Informatics and Mechanics, University of Warsaw
ERAlign: Energy-based Representation Alignment of GNNs and LLMs on Text-attributed Graphs
Jun 9, 2026Xianlin Zeng, Fan Xia, Xiangyu ChenGraph Representation Learning Cross-Modal Alignment
Postdoctoral Research Station at China RongTong Academy of Sciences Group Corporation Limited, Beijing, China · Duke University, Durham, NC, USA.
MaskAlign: Token-Subset Representation Alignment for Efficient Diffusion Training
Jun 7, 2026Lianyu Pang, Tianlin Pan, Cheng Da +5Diffusion Model Alignment Diffusion Transformer
The Hong Kong University of Science and Technology · Kuaishou Technology · University of Chinese Academy of Sciences
Representational Similarity and Model Behavior in Multi-Agent Interaction
Jun 5, 2026Yujin Potter, Seun Eisape, Shiyang Lai +6Representational Similarity Analysis Multi-Agent Collaboration
University of California, Berkeley · The University of Chicago, Knowledge Lab · Santa Fe Institute +2
Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation
Jun 4, 2026Fangbo Tu, Junhua Zhao, Chi Liu +4LLM Quantization Knowledge Distillation
PayPal, Inc.
A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting
Jun 2, 2026Peiyan Zhang, Jason XinCausal Interventions in Language Models Representation Alignment
Independent Researcher
Forgetting is Not Erasure: Recovering Latent Knowledge via Transport Keys
Jun 1, 2026Archie ChaudhuryContinual Learning Representation Alignment
Axionic Labs
PlatonicNav: Unveiling Semantic Correspondence in Navigation with Platonic Topological Maps
Jun 1, 2026Junlin Long, Zeyu Zhang, Xu Deng +5Embodied Navigation Vision-Language Navigation
1USYD · 2Maincode · 3UNSW +1
Logit Distillation on Manifolds: Mapping by Learning
May 30, 2026Yiru Yang, Junling Wang, Nishant Kumar Singh +2Teacher-Student Learning Representation Alignment
University of Zurich · ETH Zurich · Deutsche Bank Securities
Improving Visual Representation Alignment Generation with GRPO
May 30, 2026Shentong Mo, Sukmin YunDiffusion Model Alignment Diffusion Transformer
Carnegie Mellon University · Hanyang University
MIMO: Multilingual Information Retrieval via Monolingual Objectives
May 29, 2026Youngjoon Jang, Seongtae Hong, Heuiseok LimContrastive Learning Cross-Lingual Representation Alignment
Department of Computer Science and Engineering, Korea University
SLAD : Shared LoRA Adapters for Task Specific Distillation
May 28, 2026Reda Bensaid, Yassir Bendou, Vincent Gripon +1Vision Foundation Model Adaptation Adapter Tuning
IMT Atlantique, Brest, France · Polytechnique Montr´eal, Montr´eal, Canada
SIGMA: Bridging Structural and Distributional Gaps for Vision Foundation Model Adaptation
May 27, 2026Lingyu Xiong, Jinjin Shi, Xuran Xu +3Vision Foundation Model Adaptation Dense Prediction
Xiaomi Corporation
TopoAlign: Topology-Aware Visual Representation Alignment
May 25, 2026Xinyuan Yan, Rita Sevastjanova, Mennatallah El-Assady +1Data Visualization Visual Analytics
University of Utah · ETH Zürich
Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m
May 23, 2026Jordan F. McCannTransformer Interpretability Transformer
Independent Researcher
Learning Through Noise: Why Subliminal Learning Works and When It Fails
May 22, 2026Vincent C. Brockers, Roman D. Ventzke, Valentin Neuhaus +2Subliminal Learning Cross-Architecture Knowledge Distillation
Max Planck Institute for Dynamics and Self-Organization, Göttingen · Faculty of Physics, Institute for the Dynamics of Complex Systems, University of Göttingen
Representation Alignment Rests on Linear Structure
May 22, 2026Kiril Bangachev, Guy Bresler, Yury PolyanskiyLinear Representation Hypothesis Cross-Modal Alignment
Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology
Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)
May 18, 2026Michał Brzozowski, Neo Christopher ChungSparse Autoencoders Mechanistic Interpretability
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers
May 16, 2026Shaodong Xu, Zhendong Wang, Litong Gong +4Diffusion Model Alignment Diffusion Transformer
University of Science and Technology of China
Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs
May 15, 2026Vincent-Daniel Yun, Junhyuk Jo, Sai Praneeth Karimireddy +1LLM Pruning LLM Compression
University of Southern California, USA · Inha University, Republic of Korea
Characterizing Universal Object Representations Across Vision Models
May 13, 2026Florian P. Mahner, Johannes Roth, Ka Chun Lam +3Visual Representation Learning Representation Alignment
Vision and Computational Cognition Group Max Planck Institute Justus-Liebig-University Giessen · Machine Learning Core National Institute of Mental Health Bethesda, MD, USA · Department of Cognitive Science Johns Hopkins University Baltimore, MD, USA
KAST-BAR: Knowledge-Anchored Semantically-Dynamic Topology Brain Autoregressive Modeling for Universal Neural Interpretation
May 13, 2026Haoning Wang, Wenchao Yang, Shuai Shen +1EEG Decoding Multimodal Pretraining
School of Automation Science and Electrical Engineering, Beihang University, Beijing, China. · School of Biological Science and Medical Engineering, Beihang University, Beijing, China. · State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China. +1
Improving Human Image Animation via Semantic Representation Alignment
May 11, 2026Chang Liu, Mengting Chen, Yixuan Huang +5Image-to-Video Generation Video Diffusion Models
School of Artificial Intelligence, Shanghai Jiao Tong University, China · Alibaba Group, China
Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models
May 10, 2026Aojie Yuan, Zhiyuan SuLLM Interpretability Representation Geometry in Language Models
University of Southern California · 2Duke University
CrystalREPA: Transferring Physical Priors from Universal MLIPs to Crystal Generative Models
May 9, 2026Chengqian Zhang, Yucheng Jin, Duo Zhang +2Transfer Learning Crystal Structure Generation
AI for Science Institute, Beijing, China · Center for Data Science, Peking University, Beijing, China · LMAM and School of Mathematical Sciences, Peking University, Beijing, China +3

Jun 25, 2026, cs.CLJ/K move · Enter open · S save

Soft Token Alignment for Cross-Lingual Reasoning

Jiayi He, Jungsoo Park, Wei Xu, Alan Ritter

Georgia Institute of Technology

Multilingual Language Models Cross-Lingual Reasoning in Language Models LLM Fine-Tuning Cross-Lingual Representation Alignment Cross-Lingual Consistency in Language Models Representation Alignment+1

Open paper AskPDF arXiv

Multilingual large language models often produce inconsistent reasoning and answers for semantically equivalent prompts in different languages. Prior work suggests that intermediate representations can be relatively language-agnostic, but generation becomes increasingly language-specific as models commit to discrete output tokens. This is problematic because language-specific lexical choices can cause semantically equivalent reasoning paths to diverge across languages. These divergences motivate searching for a cross-lingual alignment signal that is less tied to any single vocabulary item or script. We propose SOLAR, an auxiliary objective for supervised fine-tuning that aligns soft-token representations across languages, using English as a pivot. Soft tokens are probability-weighted mixtures over the vocabulary embeddings, yielding continuous representations that can aggregate information from semantically related tokens across languages. We then align each non-English soft-token summary to its English counterpart in the shared embedding space. Across four multilingual reasoning benchmarks, SOLAR improves accuracy by up to +17.7 points over the base model and +3.8 over standard supervised fine-tuning, with the largest gains on low-resource languages. SOLAR also strengthens final-layer cross-lingual similarity and substantially reduces language-cluster separability, suggesting that aligning soft-token representations helps preserve shared semantic structure during multilingual reasoning.

Momentum

Related topics

Latest papers 82