Large Language Model Memory

Follow this topic Ask about this topic

Recent momentum

-31%

9 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Large Language Model Memory.

Period ending 2026-09-14

5 new papers

A weekly snapshot of new work published in Large Language Model Memory.

91 papers

Latest in Large Language Model Memory

Browse all topics →

CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM
May 24, 2026Yubo Li, Yidi MiaoKv-Cache Blocks Key-Value Cache Eviction
WhenLoss: Diagnosing Write and Retrieval Bottlenecks in Long-Context Memory Systems
May 23, 2026Jiangnan Yu, Kisson Songqi Lin, Jilong WuLarge Language Model Memory Long-Term Memory
The Model Is Not the Product: A Dual-Pillar Architecture for Local-First Psychological Coaching
May 23, 2026Alexander MihalceaLarge Language Model Memory Cognitive Behavioral Therapy
LLM-Metrics: Measuring Research Impact Through Large Language Model Memory
May 21, 2026Si Shen, Wenhua Zhao, Danhao ZhuReal Datacite Dois Citations
Episodic-Semantic Memory Architecture for Long-Horizon Scientific Agents
May 17, 2026Nikola MilosevicLarge Language Model Memory Episodic Experiences
NGM: A Plug-and-Play Training-Free Memory Module for LLMs
May 16, 2026Yuwen Qu, Wenhui Dong, Chenyang Si +1Large Language Model Memory N$-Grams Features
Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory
May 14, 2026Songwei Dong, Zihan Chen, Chengshuai Shi +3Large Language Model Memory Sequential
MeMo: Memory as a Model
May 14, 2026Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong +6Large Language Model Memory
Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches
May 12, 2026Shaoke Fang, Ziang Li, Wenfei Wu +3Cache Eviction
Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrieval
May 11, 2026Jeffrey FlyntLarge Language Model Memory Generative
Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
May 10, 2026Yuanyi Wang, Yifan Yang, Su Lu +9Catastrophic Forgetting Large Language Model Post-Training
Reformulating KV Cache Eviction Problem for Long-Context LLM Inference
May 8, 2026Tho Mai, Joo-Young KimKey-Value Cache Eviction Key-Value Cache
Memory as a Markov Matrix: Sample Efficient Knowledge Expansion via Token-to-Dictionary Mapping
May 5, 2026Kaustubh Pethkar, Ziyang Xiong, Zuofeng Shang +1Large Language Model Memory Autoregressive Language Models
Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
May 4, 2026Prakhar Gupta, Garv Shah, Satyam Goyal +1Large Language Model Memory Forgetting
Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
Apr 28, 2026Jiaming Yang, Chenwei Tang, Liangli Zhen +1Key-Value Cache Eviction Eviction
Evolve: A Persistent Knowledge Lifecycle for Small Language Models
Apr 25, 2026Dikran HovagimianLarge Language Model Memory Retrievers
Spatial Metaphors for LLM Memory: A Critical Analysis of the MemPalace Architecture
Apr 23, 2026Robin Dey, Panyanon ViradechaLarge Language Model Memory Long-Term Memory
SCM: Sleep-Consolidated Memory with Algorithmic Forgetting for Large Language Models
Apr 22, 2026Saish Sachin ShindeLarge Language Model Memory Catastrophic Forgetting
To Know is to Construct: Schema-Constrained Generation for Agent Memory
Apr 22, 2026Lei Zheng, Weinan Song, Daili Li +1Agent Memory Large Language Model Memory
Continuous Semantic Caching for Low-Cost LLM Serving
Apr 21, 2026Baran Atalar, Xutong Liu, Jinhang Zuo +3Large Language Model Serving Large Language Model Memory
Neural Garbage Collection: Learning to Forget while Learning to Reason
Apr 20, 2026Michael Y. Li, Jubayer Ibn Hamid, Emily B. Fox +1Catastrophic Forgetting Large Language Model Memory
AnchorMem: Anchored Facts with Associative Contexts for Building Memory in Large Language Models
Apr 19, 2026Zhanyu Shen, Sijie Cheng, Zhicheng Guo +3Large Language Model Memory Contextual
LMEB: Long-horizon Memory Embedding Benchmark
Mar 13, 2026Xinping Zhao, Xinshuo Hu, Jiaxin Xu +9Large Language Model Memory Longbench
MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning
Mar 3, 2026Jiejun Tan, Zhicheng Dou, Liancheng Zhang +3Large Language Model Memory Deep Reinforcement Learning
Understanding LoRA as Knowledge Memory: An Empirical Analysis
Mar 1, 2026Seungju Back, Dongwoo Lee, Naun Kang +4Large Language Model Memory Low-Rank Adaptation
From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching
Jan 30, 2026Zhixiang Zhang, Zesen Liu, Yuchong Xie +2Language Model Evasion Attacks Cache
Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs
Nov 25, 2025Xuyuan Liu, Shengyu Chen, Xinshuai Dong +6Multimodal Knowledge Editing Large Language Model Memory
Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning
Nov 25, 2025Zhen Zeng, Leijiang Gu, Zhangling Duan +4Large Language Model Unlearning Multi-Modal Large Language Models
BudgetMem: Training-Free Selective Memory for Cost-Efficient Long-Context Processing in Language Models
Nov 7, 2025Chandra Vamsi Krishna Alla, Harish Naidu Gaddam, Manohar Kommi +1Large Language Model Memory Large Language Model Compression
Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
Oct 21, 2025Howard Chen, Noam Razin, Karthik Narasimhan +1Catastrophic Forgetting Forgetting
Memory-Efficient FastText: A Comprehensive Approach Using Double-Array Trie Structures and Mark-Compact Memory Management
Jun 2, 2025Yimin DuN$-Grams Features Domain-Invariant Text Embeddings

May 24, 2026 · cs.LGJ/K move · Enter open · S save

CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM

Yubo Li, Yidi Miao

Carnegie Mellon University

Long-horizon LLM inference turns the key--value (KV) cache into the dominant GPU memory consumer and makes per-token attention increasingly expensive. Many common eviction policies use static recency windows or historical attention, leaving unused a signal computed on every decoding step: the model's current uncertainty. We introduce CONF-KV, a KV-cache manager that converts the next-token distribution into a scalar confidence score and uses it to choose the per-step cache budget, retaining more context when the model is uncertain and pruning aggressively when it is confident. Within each budget, tokens are ranked by a composite of accumulated attention mass and recency, while a protected recent window preserves local coherence. We combine the policy with blockwise online-softmax attention, mixed FP16/INT8 storage, and a pyramidal per-layer budget variant. Across four model families and generated lengths up to 4K, CONF-KV stays near the footprint of a fixed 512-token sliding window while remaining within 1.5--2.1 perplexity points of full KV. On Needle-in-a-Haystack up to 32K tokens, CONF-KV reaches 91.4% retrieval accuracy versus 53.8% for sliding windows and 80.6% for H2O; on 75 VisualWebArena tasks it retains 95.3% of full-KV success at 2.8 times lower peak memory.

Kv-Cache Blocks Key-Value Cache Eviction Cache LLM Inference Efficiency Large Language Model Memory Graphics Processing Unit Memory Quadratic Attention Token Budgets

Open paper Ask about it PDF