Disentangled Representation Learning

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 163

All topics

Towards Isolated Interventions via Almost Orthogonal Features in Language Models
Feb 4, 2026Moritz Miller, Florent Draye, Bernhard SchölkopfDisentangled Representation Learning Representation Learning
Max Planck Institute for Intelligent Systems, Tübingen, Germany · ETH Zurich, Switzerland · Max Planck ETH Center for Learning Systems +1
Contrastive Representation Shaping for LLM Unlearning
Jan 29, 2026Haoran Tang, Rajiv KhannaDisentangled Representation Learning Machine Unlearning
Department of Computer Science, Purdue University.
XFACTORS: Disentangled Information Bottleneck via Contrastive Supervision
Jan 29, 2026Alexandre Myara, Nicolas Bourriez, Thomas Boyer +3Variational Autoencoders Disentangled Representation Learning
ENS Paris
Dual-Prototype Disentanglement: A Context-Aware Enhancement Framework for Time Series Forecasting
Jan 23, 2026Haonan Yang, Jianchao Tang, Zhuo LiDisentangled Representation Learning Prototype Learning
National University of Defense Technology, Changsha, Hunan, China
Borrowing from anything: A generalizable framework for reference-guided instance editing
Dec 17, 2025Shengxiao Zhou, Chenghua Li, Jianhao Huang +2Disentangled Representation Learning
University of Chinese Academy of Sciences, China · Nanjing Forestry University, Nanjing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China +1
Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement
Nov 3, 2025Sekh Mainul Islam, Pepa Atanasova, Isabelle AugensteinDisentangled Representation Learning LLM Grounding
University of Copenhagen
Talk in Pieces, See in Whole: Disentangled and Hierarchical Representation Learning in Language-based Object Detection
Sep 29, 2025Sojung An, Kwanyong Park, Yong Jae Lee +1Open-Vocabulary Object Detection Disentangled Representation Learning
Korea University · University of Seoul · University of Wisconsin-Madison
Variational Learning of Disentangled Representations
Jun 20, 2025Yuli Slavutsky, Ozgur Beker, David Blei +1Variational Autoencoders Disentangled Representation Learning
Department of Statistics, Columbia University, New York City, USA · Irving Institute for Cancer Dynamics, Columbia University, New York City, USA · Department of Computer Science, Columbia University, New York City, USA +1
Representation Learning for Equivariant Inference with Guarantees
May 26, 2025Daniel Ordoñez-Apraez, Vladimir Kostić, Alek Fröhlich +3Disentangled Representation Learning Representation Learning
CSML, Italian Institute of Technology, Genova, Italy · Universit`a di Genova, Genova, Italy · University of Novi Sad, Novi Sad, Serbia +2
CDST: Color Disentangled Style Transfer for Universal Style Reference Customization
May 22, 2025Shiwen Zhang, Zhuowei Chen, Lang Chen +1Disentangled Representation Learning Image Style Transfer
ByteDance
DRESS: Disentangled Representation-based Self-Supervised Meta-Learning for Diverse Tasks
Mar 12, 2025Wei Cui, Tongzi Wu, Jesse C. Cresswell +2Disentangled Representation Learning Meta-Learning
Layer 6 AI, Toronto, Canada
Beyond Spectral Decomposition: Bayesian Contrastive Learning and its Non-negative Formulation via Factor Analysis
Jul 31, 2024Zhibin Duan, Tiansheng Wen, Yifei Wang +3Contrastive Learning Disentangled Representation Learning
Xidian University · MIT CSAIL · The University of Texas at Austin
TDDM-Melatt: A Decoupled Memory and Diffusion Framework for Generalizable Encrypted Traffic Classification
Date pendingZe Chen, Qiming Yu, Zijia Song +2Memory-Augmented Neural Networks Disentangled Representation Learning
National University of Defense Technology Hefei, Anhui, China

Feb 4, 2026, cs.LGJ/K move · Enter open · S save

Towards Isolated Interventions via Almost Orthogonal Features in Language Models

Moritz Miller, Florent Draye, Bernhard Schölkopf

Max Planck Institute for Intelligent Systems, Tübingen, Germany · ETH Zurich, Switzerland · Max Planck ETH Center for Learning Systems+1

Disentangled Representation Learning Representation Learning Orthogonal Representation Learning Mechanistic Interpretability Causal Interventions in Language Models

Open paper AskPDF arXiv

A central premise in mechanistic interpretability is that meaningful concepts in language models are represented by linear features in activation space. For such features to support reliable interventions, manipulating one feature should not substantially alter the effects of others. In practice, however, feature entanglement leads to interference such that localized interventions can have unintended downstream effects. Motivated by the \textit{Independent Causal Mechanisms} principle, we propose to constrain internal features to be almost orthogonal. We argue that this promotes modular representations amenable to causal intervention. We formalize this problem by characterizing the gap between an idealized isolated intervention and its realized effect on model outputs in terms of feature interference. We upper-bound the propagation of feature interference in terms of the self-coherence of the feature dictionary, and relate this discrepancy to an explicit orthogonality regularization on the dictionary itself. Empirically, we show that this regularization enables more isolated interventions on mathematical reasoning concepts while preserving model performance. Our code is available under \texttt{https://github.com/mrtzmllr/sae-icm}.

Momentum

Related topics

Latest papers 163