LLM Memory

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 73

All topics

Multi-Head Recurrent Memory Agents
Jul 1, 2026Jiatong Li, Samuel Yeh, Sharon LiLLM Memory Long-Context Modeling
Department of Computer Science, University of Wisconsin-Madison
Beyond Perplexity: A Behavioral Evaluation Framework for Deployment-Memory Claims in LLM Test-Time Training
Jul 1, 2026Xiangchen Song, Zhenhao Chen, Lingjing Kong +4LLM Evaluation LLM Memory
Carnegie Mellon University · MBZUAI
ISM:Self-Improving Strategy Memory for Continual Mathematical Reasoning
Jun 30, 2026Prakhar Dixit, Tim OatesContinual Learning for LLMs Memory-Augmented Language Models
Department of Computer Science, University of Maryland, Baltimore County, Baltimore, Maryland, USA
The Past Is Prologue: A Plug-in Controller for Selective Updates in Sequentially Evolving LLM Memory
Jun 30, 2026Zihan Chen, Songwei Dong, Chengshuai Shi +4Continual Learning for LLM Agents Persistent Memory for Language Models
University of Virginia · Princeton University · University of Central Florida
MMLA: Memory-Mediated Learning Architecture for Predictive Dual-State Adaptation
Jun 27, 2026Junyi Zou, Avrova DonzContinual Learning for LLMs Test-Time Adaptation
1MMLA-org · 2Communication University of China (CUC), No. 1 Dingfuzhuang East Street, Chaoyang District, Beijing 100024, China
Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One
Jun 24, 2026Alex KwonLLM Evaluation Memory-Augmented Language Models
Independent Researcher
Towards Root Memories: Benchmarking and Enhancing Implicit Logical Memory Retrieval for Personalized LLMs
Jun 22, 2026Hongxun Ding, Xiang Yu, Chengbing Wang +4LLM Personalization Memory-Augmented Language Models
University of Science and Technology of China, China
User as Engram: Internalizing Per-User Memory as Local Parametric Edits
Jun 17, 2026Bojie LiKnowledge Editing LLM Personalization
Pine AI
The Reservoir Attention Network: Cross-Pass State in Pretrained Transformers via Content-Addressable Reservoir Injection
Jun 14, 2026Emma LeonhartMemory-Augmented Neural Networks Persistent Memory for Language Models
T-Mem: Memory That Anticipates, Not Archives
Jun 13, 2026Weidong Guo, Dakai Wang, Zixuan Wang +2Persistent Memory for Language Models LLM Memory
1Tencent · 2The Hong Kong University of Science and Technology (Guangzhou)
Learning What to Remember: A Cognitively Grounded Multi-Factor Value Model for Agentic Memory
Jun 11, 2026Zhibao Chen, Qian ChengLLM Agent Memory Persistent Memory for Language Models
Huatai Securities, Nanjing, China
Substrate Asymmetry in User-Side Memory: A Diagnostic Framework
Jun 10, 2026Youwang DengRetrieval-Augmented Generation LLM Personalization
EpistemicaLab — Independent Research
Position: Hippocampal Explicit Memory Is the Cornerstone for AGI
Jun 5, 2026Sangjun ParkMemory-Augmented Language Models LLM Memory
Department of Computer Science, University of Texas at Austin, TX, USA · Cognizant AI Labs, San Francisco, CA, USA
How LoRA Remembers? A Parametric Memory Law for LLM Finetuning
May 28, 2026Ziwen Xu, Haiwen Hong, Linsong Yu +4Memorization in Language Models LLM Fine-Tuning
Zhejiang University · Alibaba Group
VikingMem: A Memory Base Management System for Stateful LLM-based Applications
May 28, 2026Jiajie Fu, Junwen Chen, Mengzhao Wang +5Memory-Augmented Language Models Persistent Memory for Language Models
Zhejiang University · China · ByteDance
Code-QA-Bench: Separating Code Reasoning from Documentation Memorization in Repository-Level QA
May 28, 2026Jun Zhang, JianYing Qu, Hanwen Du +3Software Engineering Benchmarks LLM Memory
Baidu Inc.
MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems
May 27, 2026Xinle Deng, Ruobin Zhong, Hujin Peng +15LLM Evaluation Memory-Augmented Language Models
1Zhejiang University, 2Alibaba Group
MemFail: Stress-Testing Failure Modes of LLM Memory Systems
May 26, 2026Ishir Garg, Neel Kolhe, Dawn Song +1LLM Agent Memory LLM Memory
University of California, Berkeley
Simulating Human Memory with Language Models
May 25, 2026Qihan Wang, Nicholas Tomlin, Michael Hu +2LLM Evaluation Memory-Augmented Language Models
NYU · UMass Amherst
NGM: A Plug-and-Play Training-Free Memory Module for LLMs
May 16, 2026Yuwen Qu, Wenhui Dong, Chenyang Si +1Memory-Augmented Language Models Knowledge Augmentation for Language Models
Nanjing University
Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory
May 14, 2026Songwei Dong, Zihan Chen, Chengshuai Shi +3Continual Learning for LLMs LLM Evaluation
University of Virginia · Princeton University
MeMo: Memory as a Model
May 14, 2026Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong +6Retrieval-Augmented Generation Memory-Augmented Language Models
Institute of Data Science, National University of Singapore, Singapore · Integrative Sciences and Engineering Programme, NUSGS, Singapore · Agency for Science, Technology, Research (A*STAR), Singapore +6
EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agents
May 13, 2026Jiaqi Liu, Xinyu Ye, Peng Xia +4LLM Agent Memory LLM Memory
UNC-Chapel Hill · UC Berkeley · UCSC
$δ$ -mem: Efficient Online Memory for Large Language Models
May 12, 2026Jingdi Lei, Di Zhang, Junxian Li +7Memory-Augmented Language Models Memorization in Language Models
Nanyang Technological University · Mind Lab · Fudan University +3
Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrieval
May 11, 2026Jeffrey FlyntConversational Memory LLM Memory
Independent Researcher
WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems
May 8, 2026Juan M. HuertaKV Caching Counterexample-Guided Refinement
Zinnia Tech Solutions 600 Steamboat Road Greenwich, CT 06830, USA
Continual Knowledge Updating in LLM Systems: Learning Through Multi-Timescale Memory Dynamics
May 6, 2026Andreas Pattichis, Constantine DovrolisContinual Learning for LLMs LLM Memory
The Cyprus Institute, Nicosia, Cyprus · School of Computer Science, Georgia Institute of Technology, Atlanta, GA, USA.
The Dynamic Gist-Based Memory Model (DGMM): A Memory-Centric Architecture for Artificial Intelligence
May 4, 2026Terry Dorsey, Kevin HugginsMemory-Augmented Neural Networks LLM Interpretability
Harrisburg University of Science and Technology, USA
Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance
May 3, 2026Anamika Paul Rupa, Anietie AndyMemorization in Language Models Neural Network Memorization
Howard University
From Unstructured Recall to Schema-Grounded Memory: Reliable AI Memory via Iterative, Schema-Aware Extraction
Apr 30, 2026Alex Petrov, Alexander Gusak, Denis Mukha +1LLM Memory Document Information Extraction
1xmemory

Jul 1, 2026, cs.LGJ/K move · Enter open · S save

Multi-Head Recurrent Memory Agents

Jiatong Li, Samuel Yeh, Sharon Li

Department of Computer Science, University of Wisconsin-Madison

LLM Memory Long-Context Modeling Agent Memory Management Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Recurrent memory agents extend LLMs to arbitrarily long contexts by iteratively consolidating input into a fixed-size memory window. Despite their scalability, these agents exhibit a well-documented reliability problem: end-to-end performance degrades systematically as context length grows. We diagnose this failure by decomposing performance into two factors--memory capture and memory retention--and quantitatively confirm that retention is the dominant bottleneck. Retention collapses because existing designs maintain memory as a monolithic text block, forcing every update to risk overwriting previously retained content. Motivated by this diagnosis, we propose Multi-Head Recurrent Memory (MHM), a general, training-free framework that partitions memory into independent heads governed by a stage-wise select-then-update strategy. At each step, exactly one head is selected for update while the remaining heads are structurally shielded from overwriting, shifting the burden of retention from model behavior to architectural design. As a lightweight instantiation, we introduce Least-Recently-Updated MHM (MHM-LRU), which guarantees uniform head utilization with zero additional token overhead. Extensive experiments on long-context benchmarks show that MHM-LRU substantially improves both retention and end-to-end accuracy across the 100K--1M token range, where baselines degrade sharply. On RULER-HQA at 896K tokens, MHM-LRU improves the memory retention rate from less than 30% to 73.96%. These gains generalize across model families, scales, and task types, positioning architectural optimization as a practical and cost-efficient path toward reliable long-context recurrent memory.

Momentum

Related topics

Latest papers 73