Feature Superposition

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 15

All topics

Hidden not Deleted: How Networks Suppress Entangled Features
Sep 23, 2026Akash Samanta, Manish Pratap Singh, Debasis ChaudhuriFeature Superposition Mechanistic Interpretability
Techno India University Salt Lake, Kolkata, India · DRDO Young Scientist Laboratory - CT Chennai, India
Topographic Training Concentrates Causal Circuits Without Improving Neuron Monosemanticity
Sep 21, 2026Gautam Ranka, Shubham Santosh Pandere, Aiden DsouzaFeature Superposition Vision Transformer
IvLabs, VNIT Nagpur, India.
High-probability guarantees for linear accessibility in feature superposition
Sep 10, 2026Enrico VompaFeature Superposition Neural Representation Geometry
Applied Artificial Intelligence Group Tallinn University of Technology, Estonia
A Dominant Diffuse Phase in the Sparse Autoencoder Phase Diagram
Sep 9, 2026Alexis D. PlascenciaFeature Superposition Representation Disentanglement
Scaling Interpretable Transformers with Parity Bottleneck Layers
Jul 22, 2026Andrew Mack, Kraig Yuheng Tou, Mark Henry +2Feature Superposition Transformer Interpretability
Principles of Intelligence · Independent
From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness
Jul 13, 2026Mohamed Abdessalem BalFeature Superposition Neural Representation Geometry
Independent Researcher, Algiers, Algeria
Resolving superposition in AI for interpretability and cross-modal alignment in patient-neuronal images
Jun 30, 2026Jisung Park, Seohyeon Kang, Daeun Yoo +8Feature Superposition Neural Representation Geometry
1KAIST · 2Konyang University · 4UCL Queen Square Institute of Neurology & The Francis Crick Institute +1
Sparsity, Superposition, and Forgetting: A Mechanistic Study of Representation Retention in Continual Learning
Jun 18, 2026Jan Wasilewski, Jędrzej Kozal, Michał Woźniak +1Feature Superposition Representation Learning
Rochester Institute of Technology · Rochester NY, USA · Wrocław University of Science and Technology +1
Effects of sparsity and superposition on loss in simple autoencoders
Jun 16, 2026Mriganka Basu Roy Chowdhury, Eric McLaughlin WeinerFeature Superposition Activation Sparsity
Department of Statistics, UC Berkeley · Department of Materials Science, UC Berkeley
Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models
Jun 1, 2026Alexander GuhaFeature Superposition Transformer Expressivity
Arizona State University
Probing for Representation Manifolds in Superposition
May 18, 2026Alexander ModellFeature Superposition Neural Representation Geometry
Department of Mathematics Imperial College London, U.K.
Linear-Readout Floors and Threshold Recovery in Computation in Superposition
May 2, 2026Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-CarbóFeature Superposition
VRAIN – Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de València, Valencia, Spain · Department of Economics and Social Sciences, Universitat Politècnica de València, Valencia, Spain · Department of Business Organisation, Universitat Politècnica de València, Valencia, Spain
Adjoint Inversion Reveals Holographic Superposition and Destructive Interference in CNN Classifiers
Apr 30, 2026Kaixiang ShuFeature Superposition Neural Network Interpretability
Independent Researcher · Chongqing, China
Structural Instability of Feature Composition
Apr 18, 2026Yunpeng ZhouFeature Superposition Transformer Interpretability
Whiteknights House, Whiteknights, Reading, RG6 6UR
Similarity of Neural Network Representations in Superposition
Mar 31, 2026Sunny Liu, Habon Issa, André Longon +4Feature Superposition Representational Similarity Analysis
Cold Spring Harbor Laboratory · UC San Diego · Anthropic +2

Sep 23, 2026, cs.LGJ/K move · Enter open · S save

Hidden not Deleted: How Networks Suppress Entangled Features

Akash Samanta, Manish Pratap Singh, Debasis Chaudhuri

Techno India University Salt Lake, Kolkata, India · DRDO Young Scientist Laboratory - CT Chennai, India

Feature Superposition Mechanistic Interpretability Concept Erasure Machine Unlearning

Open paper AskPDF arXiv

Concept erasure methods that operate via linear projection assume that features occupy separable subspaces. We show this assumption fails under dense superposition: when two features are forced into an antipodal pair sharing a single subspace, state-of-the-art linear erasure destroys both, not just the target. Networks trained with gradient descent instead solve this problem non-linearly, but not uniformly: they converge to one of two distinct circuit-level solutions depending on initialization, which we call mirror and shadow solutions. We map this bifurcation as a function of feature entanglement, show it reflects a stable attractor structure rather than an artifact of our setup, and use targeted causal interventions to demonstrate that both solutions leave a substantial, measurable trace of the erased feature's representation intact, recoverable through a single scalar patch rather than requiring any further training. This mirrors a failure mode recently observed empirically in LLM unlearning, where suppression rather than deletion allows forgotten knowledge to resurface; our results offer a mechanistic, causally-validated account of why that failure mode occurs.

Momentum

Related topics

Latest papers 15