LLM Memory

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 73

All topics

A Parametric Memory Head for Continual Generative Retrieval
Apr 25, 2026Kidist Amde Mekonnen, Yubao Tang, Maarten de RijkeMemory-Augmented Neural Networks LLM Memory
University of Amsterdam Amsterdam, The Netherlands
A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repair
Apr 23, 2026Milan De Koning, Ali Asgari, Pouria Derakhshanfar +1LLM Evaluation Metamorphic Testing
JetBrains Research, Amsterdam, The Netherlands · Delft University of Technology, Delft, The Netherlands
Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?
Apr 23, 2026Shivam Rawat, Lucie Flek, Florian Mai +1CoT Reasoning LLM Memory
Lamarr Institute for Machine Learning and Artificial Intelligence · Rheinische Friedrich-Wilhelms-Universität Bonn
Spatial Metaphors for LLM Memory: A Critical Analysis of the MemPalace Architecture
Apr 23, 2026Robin Dey, Panyanon ViradechaPersistent Memory for Language Models LLM Memory
OpenHub Research Chiang Mai, Thailand
Knowledge Capsules: Structured Nonparametric Memory Units for LLMs
Apr 22, 2026Bin Ju, Shenfeng Weng, Danying Zhou +2Memory-Augmented Language Models Knowledge Augmentation for Language Models
Zhejiang Angel Medical AI Technology Co., Ltd., Hangzhou, China · Miti AI Technology Co., Ltd., Hangzhou, China · China-Singapore Belt and Road Joint Laboratory on Translational Infection Biology for Diagnostics and Therapies, State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital
The Continuity Layer: Why Intelligence Needs an Architecture for What It Carries Forward
Apr 19, 2026Samuel Sameer TanguturiMemory-Augmented Language Models Persistent Memory for Language Models
Kenotic Labs
MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning
Mar 3, 2026Jiejun Tan, Zhicheng Dou, Liancheng Zhang +3Memory-Augmented Language Models Persistent Memory for Language Models
Gaoling School of Artificial Intelligence · Renmin University of China · Beijing, China
Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model
Feb 6, 2026Jacqueline He, Jonathan Hayase, Wen-tau Yih +3Constrained Decoding Language Model Decoding
University of Washington · Allen Institute for Artificial Intelligence
MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewards
Jan 9, 2026Zhiyu Shen, Ziming Wu, Fuming Lai +2Reinforcement Learning Memory-Augmented Language Models
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Tencent Inc.
Beyond Static Summarization: Proactive Memory Extraction for LLM Agents
Jan 8, 2026Chengyuan Yang, Zequn Sun, Wei Wei +1LLM Agent Memory Persistent Memory for Language Models
State Key Laboratory for Novel Software Technology, Nanjing University, China · National Institute of Healthcare Data Science, Nanjing University, China
Quantifying the Effect of Test Set Contamination on Generative Evaluations
Jan 7, 2026Rylan Schaeffer, Joshua Kazdan, Baber Abbasi +8Language Model Generation Evaluation Benchmark Contamination
Stanford Computer Science · Stanford Statistics · EleutherAI +3
Membox: Weaving Topic Continuity into Long-Range Memory for LLM Agents
Jan 7, 2026Dehao Tao, Guoliang Ma, Yongfeng Huang +1LLM Memory Long-Term Conversational Memory
Tsinghua University · Xinjiang University
Mitigating Memorization In Language Models
Oct 3, 2024Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6Memorization in Language Models LLM Fine-Tuning
University of Chicago · Argonne National Laboratory · Lawrence Berkeley National Laboratory +3

Apr 25, 2026, cs.IRJ/K move · Enter open · S save

A Parametric Memory Head for Continual Generative Retrieval

Kidist Amde Mekonnen, Yubao Tang, Maarten de Rijke

University of Amsterdam Amsterdam, The Netherlands

Memory-Augmented Neural Networks LLM Memory Generative Retrieval Catastrophic Forgetting

Open paper AskPDF arXiv

Generative information retrieval (GenIR) consolidates retrieval into a single neural model that decodes document identifiers (docids) directly from queries. While this model-as-index paradigm offers architectural simplicity, it is poorly suited to dynamic document collections. Unlike modular systems, where indexes are easily updated, GenIR's knowledge is parametrically encoded in its weights; consequently, standard adaptation methods such as full and parameter-efficient fine-tuning can induce catastrophic forgetting. We show that sequential adaptation improves retrieval on newly added documents but substantially degrades performance on earlier slices, exposing a pronounced stability-plasticity trade-off. To address this, we propose post-adaptation memory tuning (PAMT), a memory-only stabilization stage that augments an adapted model with a modular parametric memory head (PMH). PAMT freezes the backbone and attaches a product-key memory with fixed addressing. During prefix-trie constrained decoding, decoder hidden states sparsely query PMH to produce residual corrections in hidden space; these corrections are mapped to score adjustments via the frozen output embedding matrix, computed only over trie-valid tokens. This guides docid generation while keeping routing and backbone parameters fixed. To limit cross-slice interference, PAMT updates only a fixed budget of memory values selected using decoding-time access statistics, prioritizing entries frequently activated by the current slice and rarely used in prior sessions. Experiments on MS MARCO and Natural Questions under sequential, disjoint corpus increments show that PAMT substantially improves retention on earlier slices with minimal impact on retrieval performance for newly added documents, while modifying only a sparse subset of memory values per session.

Momentum

Related topics

Latest papers 73