LLM Memory

LLM: Large Language Model

The field asks how language models can carry useful information beyond the active context, from facts encoded in model weights to persistent memories for agents. Approaches include retrieval-augmented stores built from summaries, facts, graphs, or key-value records; recurrent or attention-coupled memory modules; and fine-tuning or editing model parameters. Benchmarks increasingly test long conversations, multi-hop and temporal questions, streaming updates, personalization, and forgetting rather than only one-shot recall. A central tension is compressing and updating memory cheaply while preserving relevant detail and the ability to correct stale or misleading information; retrieved memories can also distort reasoning.

Latest papers

95 papers8 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Memory Tax: Dissecting the Net Value of Agent Memory
Oct 25, 2026Xuan He, Sining Zhoubian, Ruozhen Yang +8LLM Agent Memory LLM Memory
University of Illinois Urbana Champaign · Tsinghua University · Renmin University of China +1
The Geometry of Agent Memory: Forecasting Retrieval-Bound Memory Failures, and Characterizing Its Limits
Oct 19, 2026Iliya Zhechev, Radoslav Ralev, Chris Coleman +4LLM Memory Conversational Memory
Redis
Understanding and Mitigating Inference-Time Overreliance Using Agentic Memory
Oct 5, 2026Luoxi Tang, Yuqiao Meng, Nilesh Auradkar +3Agent Memory LLM Memory
Binghamton University, State University of New York, Binghamton, NY, USA · University of Iowa, Iowa City, IA, USA · Penn Medicine, University of Pennsylvania, Philadelphia, PA, USA
MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agents
Oct 5, 2026Haozhen Zhang, Haodong Yue, Quanyu Long +6LLM Agent Memory LLM Memory
Nanyang Technological University · Tsinghua University · University of Illinois Urbana-Champaign
Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval
Oct 1, 2026Arman Behnam, Binghui WangLLM Memory Causal Inference
Department of Computer Science Illinois Institute of Technology Chicago, IL
Learning What to Remember: Long-horizon Counterfactual Memory Optimization
Sep 29, 2026Jiaming Tang, Mingyan Liu, Armin SarabiMemory-Augmented Language Models Persistent Memory for Language Models
Department of Electrical Engineering and Computer Science, University of Michigan
FactorEngram: Factorized N-gram Memory with Basis-Level Gating for Language Models
Sep 28, 2026Bowen Yang, Jingbo Zhou, Qinghong Miao +1Memory-Augmented Language Models LLM Memory
Large Model Frontier Research Department, Baidu Inc., China · Nanyang Technological University
DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory
Sep 28, 2026Xuan Truong Nguyen, Tien Son Pham, Tuan Duc Chu +2LLM Serving LLM Inference
Department of Next Generation Semiconductor Convergence and Open Sharing System (COSS), Seoul National University, Seoul 08826, South Korea · Van-Lang Institute of Semiconductor Technology (VIST), Hanoi, Vietnam · Efficient Computation Research Group, Moreh Vietnam +1
Positions Are Not Facts: The Mismatch Between KV Caches and Memory
Sep 27, 2026Changhai Zhou, Yuhua Zhou, Shiyang Zhang +5Knowledge Editing Knowledge Conflicts in Language Models
Fudan University · Baidu · Zhejiang University +2
Grounding Memory Summarization in Utility Intent
Sep 27, 2026Zhenyu Lei, Mingjia Shi, Xingbo Fu +3Conversational Memory LLM Memory
University of Virginia · Northeastern University
LSTMem: Hierarchical Long Short-Term Online Memory for Large Language Models
Sep 27, 2026Xianglong Shi, Ruijie Yang, Sirui Zhao +4Memory-Augmented Language Models Persistent Memory for Language Models
University of Science and Technology of China
Complementary Roles of Activation and Parametric Memory in Few-Shot Learning
Sep 23, 2026Miaohe Niu, Runsong Zhao, Xinyu Liu +5LLM Memory Few-Shot Learning
Northeastern University, Shenyang, China · NiuTrans Research, Shenyang, China
Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents
Sep 23, 2026Yefan Zhou, Yang Li, Zeyu Leo Liu +2LLM Agent Memory Agent Memory Management
Salesforce AI Research
SpeakerMem-R1: Speaker-Centered Dual-Track Memory for Multi-Party Dialogue
Sep 22, 2026Haobo Zheng, Tan Tang, Yan Chen +2Conversational Memory LLM Memory
State Key Lab of CAD&CG, Zhejiang University
Pull: Lazy Materialization of Working Memory for Stateful LLM Conversations
Sep 13, 2026Jiangang ChenLLM Inference Efficiency LLM Memory
Chengdu Beiluoshimen Technology Co., Ltd.
BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference
Sep 11, 2026Anish Saxena, Jae Hyung Ju, Hritvik Taneja +4LLM Inference Efficiency LLM Inference Acceleration
Georgia Tech Atlanta, USA · Nvidia Research Westford, USA · Nvidia Research & Stanford Santa Clara, USA
Residual Vector-based Reconstruction as Long-Context Recall Regardless of Context Window Size
Sep 11, 2026MyungHoon Ryu, XinYu Piao, Jong-Kook KimLLM Memory Long-Context Modeling
Korea University South Korea
The Cost of Compression: A Rate-Distortion Limit on Factual Hallucination
Sep 10, 2026Xi Wang, Shijia Xu, Rongfeng GuoMemorization in Language Models Hallucination in Language Models
Hefei Institutes of Physical Science, Chinese Academy of Sciences, China · University of Science and Technology of China, China · Chongqing University, China +1
EvalMem: An Operation-Level Diagnostic Framework for Long-Term Memory Systems
Sep 3, 2026Zeyu Liu, Jian Zhong, Rongduo Han +12LLM Memory LLM Agent Memory
Nankai University · vivo AI Lab · Tsinghua University +1
What It Costs to Compose, Rebuild, and Correct Precomputed Memory
Aug 31, 2026Asa ShepardLLM Inference Efficiency Memory-Augmented Language Models
Department of Computer Science, Williams College
MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use
Aug 20, 2026Mengru Wang, Haozhe Luo, Zhenqian Xu +6Memory-Augmented Language Models LLM Memory
Zhejiang University · National University of Singapore · Northeastern University +2
Mental Model Management: An Operator-Based Framework for LLM Memory
Aug 16, 2026Oliver KramerPersistent Memory for Language Models LLM Memory
Computational Intelligence Lab Department of Computer Science University of Oldenburg, Germany
Mitigating Over-Personalization in LLMs via Structured Memory
Aug 8, 2026Hakeem Hannoon, Andrew Zhao, Mihir Narayan +2LLM Sycophancy Persistent Memory for Language Models
University of Saskatchewan · University of California–Irvine · University of Wisconsin–Madison +2
PsychoAgent: An Affect-Sensitive Cognitive Architecture for Conflict-Aware Memory in LLM Agents
Aug 7, 2026Mohammad Amanlou, Parham Abed Azad, Farbod Davoodi +3Cognitive Architectures for AI Agents LLM Agent Memory
School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran. · Department of Computer Engineering, Sharif University of Technology, Tehran, Iran. · Missouri University of Science and Technology, Rolla, MO, USA. +1
From Test-Time Scaling to Reusable Memory: Measuring Crystallization in Text-to-SQL
Aug 7, 2026Jiaqian Wang, Yutao Qi, Wenjin Hou +2LLM Memory Text-to-SQL
Xidian University · Xidian University Xi’an, Shaanxi, China · Shanghai Jiao Tong University +1
Runtime Observability for Heterogeneous Attention Memory
Aug 6, 2026Fanzhe Wei, Li Liu, Ziyang Wang +1KV Caching LLM Auditing
Metask Lab
LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inference
Aug 3, 2026Zhichen Liu, Ruihan Sun, Hengjie Yang +4Memory-Augmented Language Models LLM Inference
Southern University of Science and Technology · Xidian University
V-Mem: Modality-Routed Retrieval for Long-Term Multimodal Agentic Memory
Aug 2, 2026Dingyi Kang, Dongming Jiang, Yi Li +2Multimodal IR Multimodal Memory
Department of Computer Science, The University of Texas at Dallas · Department of Electrical and Computer Engineering, University of Florida
$Σ$ -Mem: An Online Reliability Memory for LLM-based Multi-Agent Systems
Jul 30, 2026Peilin Feng, Suorong Yang, Soujanya PoriaAI Agent Reliability Multi-Agent LLM Systems
1DeCLaRe Lab, Nanyang Technological University
ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models
Jul 29, 2026Ruxi Gu, Zhenliang Zhang, Zilei Wang +1Knowledge Editing Continual Learning for LLMs
Department of Automation, University of Science and Technology of China, Hefei, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China.

Oct 25, 2026J/K move · Enter open · S save

Memory Tax: Dissecting the Net Value of Agent Memory

Xuan He, Sining Zhoubian, Ruozhen Yang, Yuhao Wen+7

University of Illinois Urbana Champaign · Tsinghua University · Renmin University of China+1

LLM Agent Memory LLM Memory Memory Agent Benchmarks

Open paper AskPDF arXiv

Memory is increasingly used in long-horizon agents, yet the net value of building memory over direct retrieval from raw history remains largely unclear. Through a large-scale evaluation across multiple benchmarks and model backbones, we find many dedicated memory systems underperform naive RAG in many cases, indicating a negative net value. Taking RAG as a reference, we dissect the net value of memory into the gains and losses over direct retrieval, termed memory benefit and memory tax. These gains and losses arise because memory process changes how the raw information is organized and accessed. We view such changes as the double-edged deviations from the raw history. They are originally designed and introduced to benefit the task, for example by reorganizing facts or distilling preferences, but some may also remove or obscure evidence, giving rise to memory tax. Since memory benefit reflects the expected gains of its design, we then focus more on the overlooked memory tax, study how it arises from the system and evaluation side, and generalize the concept to more application scenarios of memory. We hope the notion of memory tax can provide a new lens and shift more attention from standalone performance to the net value of memory, informing more reliable memory designs and evaluations. We will open-source the code and results to the community. Project Page: https://hexuan21.github.io/MemoryTax/