Associative Memory

Follow this topic Ask about this topic

Associative memory asks how a system can store information so that a cue brings back a related pattern, fact, or sequence, a problem central to neural memory and efficient AI. Classical Hopfield networks, dense and kernel associative memories, and Hebbian learning use attractor dynamics or similarity-based retrieval; newer recurrent and linear-attention models add sparse key-value stores, while agent systems use linked episodic memories. Theory examines capacity, retrieval basins, stability, and interference, while experiments test noisy pattern completion, long-range recall, language-model memory, and hardware implementations. A persistent tension is that more storage does not guarantee reliable, selective recall: finite state and overlapping representations can cause interference, forgetting, and false associations, even as explicit retrieval improves access at added cost.

Latest papers

120 papers16 papers in the last four weeks, down 16% on the four weeks before. 0.1% of all new papers.

Blurry Window Attention
May 31, 2026Axel Laborieux, Christos Sourmpis, Juan Gabriel Kostelec +1Long-Context Language Modeling Linear Attention
Huawei, Zurich, Switzerland · Huawei Advanced Computing and Storage Lab, Shenzhen, China
Memory by Design: Bayesian Layers for Sequence Modeling
May 29, 2026Matthew Dowling, Hyungju Jeon, Cristina Savin +1Long-Context Retrieval Bayesian Filtering
Champalimaud Research, Champalimaud Foundation, Portugal · Center for Neural Science, New York University, USA · Center for Data Science, New York University, USA
Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks
May 23, 2026Aakash Kumar, Anatoly Khina, Frederik Mallmann-Trenn +1Dynamical Systems Hopfield Networks
COATI, CNRS, Inria, I3S, Université Côte d’Azur, France · School of Electrical and Computer Engineering, Tel Aviv University, Israel · Department of Informatics, King’s College London, UK +1
Bridging Silicon and the Hippocampus: Algebro-Deterministic Memory "VaCoAl" as a Substrate for Vector-HaSH and TEM
May 15, 2026Hiroyuki Chuma, Kanji Otsuka, Yoichi SatoHyperdimensional Computing Associative Memory
Institute of Innovation Research, Hitotsubashi University, Kunitachi, Tokyo 186-8603, Japan · Meisei University, Hino, Tokyo, Japan · Shuhari System, Tokyo, Japan
OSDN: Improving Delta Rule with Provable Online Preconditioning in Linear Attention
May 13, 2026Chenyu Zhou, Hongpei Li, Yuerou Liu +3Transformer Attention Linear Attention
Shanghai Jiao Tong University · Northwestern University · Huazhong University of Science and Technology +1
PaMM: Periodic Motif Memory for Atomistic Models with an Explicit Local-Structure Interface
May 13, 2026Ryan DongMemory-Augmented Neural Networks Machine Learning Interatomic Potentials
Independent Research
Does Engram Do Memory Retrieval in Autoregressive Image Generation?
May 13, 2026Jinghao Wang, Qiyuan He, Chunbin Gu +1Memory-Augmented Neural Networks Image Generation
1The Chinese University of Hong Kong · 2National University of Singapore
$δ$ -mem: Efficient Online Memory for Large Language Models
May 12, 2026Jingdi Lei, Di Zhang, Junxian Li +7Memory-Augmented Language Models Memorization in Language Models
Nanyang Technological University · Mind Lab · Fudan University +3
A Fast and Energy-Efficient Latch-Based Memristive Analog Content-Addressable Memory
May 12, 2026Paul-Philipp Manea, Aishwarya Natarajan, Jim Ignowski +2Compute-in-Memory Energy-Efficient ML
PGI-14, Forschungszentrum Jülich, Jülich, Germany · RWTH Aachen University, Aachen, Germany · Hewlett Packard Enterprise, Fort Collins, CO, USA
Variational Linear Attention: Stable Associative Memory for Long-Context Transformers
May 11, 2026Vishal Pandey, Gopal SinghEfficient Attention Linear Attention
Independent Researcher London, UK · Metriqual Athens, GR
Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
May 11, 2026Alessio Giorlandino, Sebastian Goldt, Antoine MaillardStatistical Learning Theory Associative Memory
1International School of Advanced Studies (SISSA), Trieste, Italy · 2INRIA Paris & DI ENS, PSL University, Paris, France
Hyperparameter Transfer for Dense Associative Memories
May 11, 2026Roi Holtzman, Dmitry Krotov, Boris HaninHyperparameter Transfer Associative Memory
Rudolf Peierls Centre for Theoretical Physics, University of Oxford, Oxford OX1 3PU, UK · Dynamical Mind, IBM Research · Princeton
Context-Gated Associative Retrieval: From Theory to Transformers
May 8, 2026Moulik Choraria, Argyrios Gerogiannis, Vidhata Jayaraman +2Transformer Interpretability In-Context Learning
UIUC · University of Minnesota, Twin Cities · Stony Brook University
Echo: KV-Cache-Free Associative Recall with Spectral Koopman Operators
May 7, 2026Anupama Sridhar, Alexander JohansenState Space Language Models Koopman Operator Learning
California, USA · Stanford University Computer Science California, USA
Adaptive Memory Decay for Log-Linear Attention
May 7, 2026Yaxita Amin, Helen Zichen Li, Mengfan Zhang +1Long-Context Modeling Linear Attention
University of Maryland
Efficient event-driven retrieval in high-capacity kernel Hopfield networks
May 7, 2026Akira TamamoriHopfield Networks Neuromorphic Computing
Faculty of Information Science, Aichi Institute of Technology 1247 Yachigusa, Yakusa-cho, Toyota-shi, Aichi 470-0392, Japan
Sharp Capacity Thresholds in Linear Associative Memory: From Winner-Take-All to Listwise Retrieval
May 6, 2026Nicholas Barnfield, Juno Kim, Eshaan Nichani +2Information Retrieval Associative Memory
Harvard University · University of California, Berkeley · Princeton University
Continual Knowledge Updating in LLM Systems: Learning Through Multi-Timescale Memory Dynamics
May 6, 2026Andreas Pattichis, Constantine DovrolisContinual Learning for LLMs LLM Memory
The Cyprus Institute, Nicosia, Cyprus · School of Computer Science, Georgia Institute of Technology, Atlanta, GA, USA.
The Impossibility Triangle of Long-Context Modeling
May 6, 2026Yan ZhouLong-Context Modeling Associative Memory
School of Mathematics and Statistics Changsha University of Science and Technology Changsha, Hunan 410114, China
Benchmarking local Hebbian learning rules for memory storage and prototype extraction
May 1, 2026Anders Lansner, Andreas Knoblauch, Naresh B Ravichandran +1Hebbian Learning Recurrent Neural Networks
1KTH Royal Institute of Technology, EECS, Stockholm · 2Stockholm University, Dept. of Mathematics, Stockholm · 3KEIM-Institute, Albstadt-Sigmaringen University, Germany +1
Geometric and dynamical analysis of attractor boundaries and storage limits in kernel Hopfield networks
May 1, 2026Akira TamamoriDynamical Systems Neural Representation Geometry
Faculty of Information Science, Aichi Institute of Technology 1247 Yachigusa, Yakusa-cho, Toyota-shi, Aichi 470-0392, Japan
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Apr 29, 2026Bao Pham, Mohammed J. Zaki, Luca Ambrogioni +2Memorization in Generative Models Neural Network Memorization
1Rensselaer Polytechnic Institute (RPI) · 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University · 3Dynamical Mind +1
Emergent Self-Attention from Astrocyte-Gated Associative Memory Dynamics
Apr 28, 2026Arnau Vivet, Alex ArenasSoftmax Attention Dynamical Systems
1Departament d’Enginyeria Inform`atica i Matem`atiques, Universitat Rovira i Virgili, 43007 Tarragona, Spain · 2Complexity Science Hub Vienna, Metternichgasse 8, 1030 Vienna, Austria
Learning Hippo: Multi-attractor Dynamics and Stability Effects in a Biologically Detailed CA3 Extension of Hopfield Networks
Apr 22, 2026Daniele Corradetti, Renato CorradettiDynamical Systems Neural Processes
Grupo de Física Matemática Instituto Superior Técnico Av. Rovisco Pais, 1049-001 Lisboa, Portugal · Dipartimento di Neuroscienze, Psicologia, Area del Farmaco e Salute del Bambino (NEUROFARBA) Università di Firenze, Italy22
Quantization robustness from dense representations of sparse functions in high-capacity kernel associative memory
Apr 22, 2026Akira TamamoriNeural Representation Geometry Hopfield Networks
Faculty of Information Science, Aichi Institute of Technology 1247 Yachigusa, Yakusa-cho, Toyota-shi, Aichi 470-0392, Japan
To Know is to Construct: Schema-Constrained Generation for Agent Memory
Apr 22, 2026Lei Zheng, Weinan Song, Daili Li +1Schema-Constrained Generation Long-Term Conversational Memory
UnionPay
FG $^2$ -GDN: Enhancing Long-Context Gated Delta Networks with Doubly Fine-Grained Control
Apr 21, 2026Pingwei Sun, Yuxuan Hu, Jianchao Tan +6Long-Context Modeling Linear Attention
Meituan · Beijing, China
HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents
Apr 18, 2026Jinchang Zhu, Jindong Li, Cheng Zhang +2Memorization in Language Models Hebbian Learning
The Hong Kong University of Science and Technology (Guangzhou) · Jilin University · The Chinese University of Hong Kong
Generic Triple-Latent Compression with Gated Associative Retrieval
Apr 17, 2026Liu XiaoMemory-Augmented Language Models Autoregressive Language Modeling
SRMU: Relevance-Gated Updates for Streaming Hyperdimensional Memories
Apr 16, 2026Shay Snyder, Andrew Capodieci, David Gorsich +1Hyperdimensional Computing Vector Symbolic Architectures
George Mason University · George Mason University, USA · Neya Robotics +3

May 31, 2026J/K move · Enter open · S save

Blurry Window Attention

Axel Laborieux, Christos Sourmpis, Juan Gabriel Kostelec, Qinghai Guo

Huawei, Zurich, Switzerland · Huawei Advanced Computing and Storage Lab, Shenzhen, China

Long-Context Language Modeling Linear Attention Attention Mechanisms Associative Memory

Open paper AskPDF arXiv

The Softmax Attention operation in Transformer language models has a quadratic complexity in the sequence length and a growing state size in the form of KV cache, which becomes a bottleneck in long context scenarios. To overcome this limitation, alternative architectures with linear complexity and finite state size have been introduced, such as State-Space Models (SSMs), Linear Attention (LA), and Attention with Bounded-memory Control (ABC). Though linear models achieve similar language perplexity as Transformers, they are still behind in tasks which require retrieval or recall of specific information. In this work, we introduce Blurry Window Attention (BLA) a novel ABC method inspired by SSMs. BLA stores a frequency window from which a blurry KV history is reconstructed via interpolation using Dirichlet kernels. BLA can be understood as a generalization of Sliding Window Attention (SWA) depending on the Dirichlet kernels resolution or as a special case of the Gated Slot Attention (GSA), where the decay factor is implemented with Dirichlet kernels. We describe in details the theory and efficient implementation of BLA. On the Multi-Query Associate Recall (MQAR) synthetic task, we show that the state efficiency of BLA is 8

\times

better than SWA and is competitive with popular linear attention models, and in the RegBench synthetic task, only BLA and SWA improve their performance as the state size grows among the linear models we tested.