Linear Representation Hypothesis

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 12

All topics

LinSlot: Exploiting Linear Representation hypothesis for unsupervised attribute discovery from slot based object representation
Oct 7, 2026Sanket Gandhi, Utkarsh Giri, Varun Subramanium +2Disentangled Representation Learning Unsupervised Learning
Yardi School of AI, IIT Delhi · Department of Computer Science and Engineering, IIT Delhi
Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?
Sep 28, 2026Teodor Chiaburu, Franz Motzkus, Frank Haußer +1Linear Representation Hypothesis Sparse Autoencoders
Fraunhofer SIT · ATHENE National Research Center for Applied Cybersecurity · AUMOVIO +3
The Linear Representation Hypothesis Needs a Group Action
Sep 22, 2026Louie Hong Yao, Yuhao Li, Shengchao LiuLinear Representation Hypothesis Neural Network Interpretability
The Chinese University of Hong Kong
The Answer-Basin Representation Hypothesis: We Are Not Probing or Steering Concepts
Sep 21, 2026Manjiang Yu, Hongji Li, Zihan Wang +5LLM Interpretability Linear Representation Hypothesis
The University of Queensland · Mohamed bin Zayed University of Artificial Intelligence · Institute of Science Tokyo
How are linear representations learned? Exact solutions to the dynamics of abstraction
Jul 9, 2026William W. Yang, Andrew M. Saxe, Peter E. LathamLinear Representation Hypothesis Deep Linear Networks
Gatsby Computational Neuroscience Unit, University College London, London W1T 4JG, United Kingdom. · Sainsbury Wellcome Centre, University College London, London W1T 4JG, United Kingdom.
Closing the Loop: PID Feedback Control for Interpretable Activation Steering in Symbolic Music Generation
Jun 17, 2026Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas +2Transformer Interpretability Controllable Music Generation
Athens University of Economics and Business, Athens, Greece · Orfium Research, Athens, Greece · Hellenic Mediterranean University, Chania, Greece +2
Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models
Jun 1, 2026Alexander GuhaFeature Superposition Transformer Expressivity
Arizona State University
Representation Alignment Rests on Linear Structure
May 22, 2026Kiril Bangachev, Guy Bresler, Yury PolyanskiyLinear Representation Hypothesis Cross-Modal Alignment
Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology
Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions
May 11, 2026Andrew Lee, Fernanda Viégas, Martin WattenbergTransformer Interpretability Neural Representation Geometry
Harvard University · Google DeepMind* · *Work done at Harvard.
The Cylindrical Representation Hypothesis for Language Model Steering
May 3, 2026Lang Gao, Jinghui Zhang, Wei Liu +7Language Model Steering LLM Interpretability
Department of Natural Language Processing, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Computer Science, National University of Singapore, Singapore
Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models
Feb 27, 2026Arnas Uselis, Andrea Dittadi, Seong Joon OhNeural Representation Geometry Visual Representation Learning
Tübingen AI Center, University of Tübingen · Helmholtz · Technical University of Munich
Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilities
Jan 29, 2026Tien Dang, The-Hai Nguyen, Dinh Mai Phuong +5Linear Representation Hypothesis Controllable Text Generation
Japan Advanced Institute of Science and Technology · Monash University · Quantum AI and Cyber Security Institute, FPT Corporation +1

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

LinSlot: Exploiting Linear Representation hypothesis for unsupervised attribute discovery from slot based object representation

Sanket Gandhi, Utkarsh Giri, Varun Subramanium, Rohan Paul+1

Yardi School of AI, IIT Delhi · Department of Computer Science and Engineering, IIT Delhi

Disentangled Representation Learning Unsupervised Learning Object-Centric Representation Learning Linear Representation Hypothesis

Open paper AskPDF arXiv

This paper studies the problem of learning disentangled representations of objects and their attributes from raw, unstructured image data. Slot-based methods have shown considerable success in unsupervised learning of object representations from images. Block-slot attention-based methods extend this framework to attribute representations by assuming a uniform factorization of object representations into attributes, which may be suboptimal and consequently limit the quality of the learned representations. We therefore investigate a framework for jointly discovering object and attribute representations. Our key contribution is leveraging the Linear Representation Hypothesis (LRH), which postulates that composable concepts can be represented as linearly additive subspaces in slot representations. Based on this insight, we propose a probabilistic model connecting images, slots (objects), and blocks (attributes). We present an architecture that leverages block attention to connect attribute representations to slots and incorporates LRH in both object and attribute representation spaces. This architecture effectively optimizes the Evidence Lower Bound (ELBO) of the proposed graphical model. Our experiments demonstrate (i) effective discovery of disentangled object and attribute representations, (ii) empirical evidence for LRH in slot space, and (iii) the ability to perform image editing owing to the disentangled and interpretable nature of the learned representations. Our experiments on multiple datasets demonstrate improvements in DCI scores over state-of-the-art methods.

Momentum

Related topics

Latest papers 12