cs.AINov 16, 2026

KV-LTM: Persistent Per-Session KV Caches as Long-Term Memory for Personal Assistants

Authors: Ziv Kfir, Ido Kfir

Abstract

Personal assistants accumulate months of conversations, but today's long-term memory stacks put derivation or selection between a question and its evidence. In a system-level comparison on LongMemEval, two commercial assistants scored well below the same model reading the history offline [1], consistent with, but not isolating, content loss between write and read. We propose KV-LTM: a memory tier storing each session's raw text plus its KV cache under a fixed reader model, answering by restoring cached sessions and appending the question. KV-LTM is complementary to, not a replacement for, retrieval and indexing. Text is the memory of record; the cache is a recompute-free prefill accelerator, degrading gracefully to text. We specify the design: storage record, budget-derived retention, append-only read path, and bounded aggregation. We size it analytically: across pinned BF16 full-attention (GQA) and latent-cached MLA configurations, per-token state spans approximately 31K-328K bytes (the pinned KDA+MLA hybrid differs: approximately 8.1K bytes/token plus a fixed per-session recurrent term), giving approximately 0.34-3.6 TB per user-year at a declared 30K-token/day scenario, against approximately 49 MB of raw text; per-query cost is modelled as regime-conditional transfer, attention, decode, aggregation, and compute. We define an evaluation protocol needing no persistent-cache implementation for logical QA evaluation: per-session prefill emulates valid cache reuse under the same model/input-state contract and ideal arithmetic, with named arms, controlled baselines, and a preregistered reader-model gate; a separate minimal real-cache replay tests numerical equivalence and measures the cost axis, with outcomes not assumed. KV-LTM trades storage for avoided re-prefill compute and write-time fidelity, within a declared retention budget.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference

    Mar 27, 2026Gradwell Dzikanyanga, Weihao Yang, Hao Huang +4KV-Cache ManagementLong-Context Language Model Inference

  2. KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse

    Oct 9, 2026Taiga Ikeda, Daisuke Miyashita, Jun DeguchiKV-Cache ReuseKV-Cache Management

  3. Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

    Oct 6, 2026Hochan Son, Kyungdoe Han, Jaehan Koh +3LLM Inference EfficiencyMulti-Agent LLM Systems