Dictionary Learning

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 16

All topics

Inference and learning in sparse autoencoders as natural gradient flow
Oct 5, 2026Hadi Vafaii, Tejas Rao, David Chanin +6Gradient Descent Sparse Autoencoders
Astera Institute · UC Berkeley · UCL +2
Iterative Atom Refinement: A Monotonicity Principle for Dictionary Learning
Sep 20, 2026Alexander Christie, Miguel Moscoso, Alexei Novikov +2Sparse Recovery Dictionary Learning
Department of Mathematics, Stanford University, Stanford, CA 94305, USA · Department of Mathematics, Universidad Carlos III de Madrid, Leganés, Madrid 28911, Spain · Department of Mathematics, Pennsylvania State University, University Park, PA 16802, USA +1
A Dominant Diffuse Phase in the Sparse Autoencoder Phase Diagram
Sep 9, 2026Alexis D. PlascenciaFeature Superposition Representation Disentanglement
Honest Physical-Support Inference after Latent Dictionary Learning: Collision Singularities and Minimax Resolution
Jul 18, 2026Guan-Ju PengSparse Recovery Dictionary Learning
Institute of Data Science and Information Computing, National Chung Hsing University.
Residual-Guided Dictionary Learning for Spectrally Accurate Koopman Approximation
Jun 27, 2026George Coote, Matthew J. ColbrookSpectral Regularization Koopman Operator Learning
Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Road, CB3 0WA, United Kingdom
On the Sparsity-Storage-Accuracy Tradeoff in Parsimoniously Activated Dictionary Learning
Jun 21, 2026Zihui Zhao, Yuanbo Tang, Yang LiStructured Sparsity Dictionary Learning
Tsinghua University, Institute of Data and Information Shenzhen Key Laboratory of Ubiquitous Data Enabling
Size Doesn't Matter: Cosine-Scored Sparse Autoencoders
Jun 13, 2026Silen Naihin, Lev StamblerRepresentation Learning Sparse Autoencoders
Experiential Labs · Tear Labs
How Optimality Structures Sparse Dictionaries: A Theory for Understanding SAE Representations
Jun 1, 2026William DorrellRepresentation Learning Sparse Autoencoders
Kempner Institute · Harvard University
Ablating Archetypes: The Stability of Archetypal SAEs is an Artifact of Initialization and Metric Design
Jun 1, 2026Michał Brzozowski, Neo Christopher ChungSparse Autoencoders Mechanistic Interpretability
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
Toward Identifiable Sparse Autoencoders
May 29, 2026Walter Nelson, Theofanis Karaletsos, Francesco LocatelloParameter Identifiability Sparse Autoencoders
Institute of Science and Technology Austria · 2Pyramidal Inc. · and Achira Inc., USA.
Learning fMRI activations dictionaries across individual geometries via optimal transport
May 20, 2026Sonia Mazelet, Rémi Flamary, Bertrand ThirionLearning to Optimize Geometric Representation Learning
CMAP, Ecole Polytechnique · Palaiseau, France · Mind, Inria-Saclay
Dictionary learning for Kernel EDMD
Apr 28, 2026Erik Lien Bolager, Boumediene Hamzi, Houman Owhadi +2Koopman Operator Learning Nonlinear System Identification
School of Computation, Information and Technology, Munich Data Science Institute, Munich Center for Machine Learning, Technical University of Munich, Germany · Department of Computing and Mathematical Sciences, Caltech, USA · The Alan Turing Institute, UK +1
Diverse Dictionary Learning
Apr 19, 2026Yujia Zheng, Zijian Li, Shunxing Fan +2Latent Variable Models Representation Identifiability
CMU · MBZUAI · NYU
Data-Driven Variational Basis Learning Beyond Neural Networks: A Non-Neural Framework for Adaptive Basis Discovery
Apr 17, 2026Andrew KirulutaRepresentation Learning Dictionary Learning
UC Berkeley, CA
Multi-Dictionary Learning for Low Rank Sparse Coding
Sep 12, 2025Boya Ma, Abram Magner, Maxwell McNeil +1Representation Learning Sample Complexity
University at Albany – SUNY, Computer Science, Albany, NY, USA
Boosting Adversarial Robustness and Generalization with Dictionary Structure
Feb 2, 2025Zhichao Hou, Weizhi Gao, Hamid Krim +2Neural Network Generalization Adversarial Robustness
Department of Computer Science, North Carolina State University, Raleigh, USA.

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

Inference and learning in sparse autoencoders as natural gradient flow

Hadi Vafaii, Tejas Rao, David Chanin, Thomas Fel+5

Astera Institute · UC Berkeley · UCL+2

Gradient Descent Sparse Autoencoders Mechanistic Interpretability Dictionary Learning Sparse Coding

Open paper AskPDF arXiv

Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. These challenges involve both inferring which features explain an input and learning the dictionary that represents them. Here, we unify inference and dictionary learning as natural-gradient flows on a shared variational free energy. We instantiate this framework as BeFOND, an encoder-free sparse coding model with closed-form inference and learning dynamics. We show how recurrent explaining away reduces interference between overlapping features, while Fisher preconditioning can compensate for the slow learning of rare features. On synthetic data, BeFOND improves dictionary recovery and rare-feature detection, with a growing advantage over amortized baselines as superposition increases. On language-model activations, it improves single-feature concept detection and selective intervention, outperforming pretrained reference SAEs with substantially less training data. Its feature quality continues to improve with dictionary width, whereas the evaluated baselines largely plateau. Together, these results show how improving inference and learning within a unified probabilistic framework can make better use of data and dictionary capacity to interpret and intervene on neural representations.

Momentum

Related topics

Latest papers 16