CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters

Yuhang Cao; Yanzhou Mu; Chunrong Fang; Zhenyu Chen

cs.LGSep 25, 2026

CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters

Authors: Yuhang Cao, Yanzhou Mu, Chunrong Fang, Zhenyu Chen

Organizations: Nanjing University

Abstract

Large language models rely on KV caching to reduce repeated prefill computation in long context and interactive applications. As lightweight adapters evolve, cached states reflect earlier versions, so stale reuse distorts current model outputs, while complete affected suffix recomputation restores fidelity at substantial cost. We seek minimal recomputation that recovers current adapter behavior. Existing systems track token, context, or stable adapter identity, but neither represent caches from earlier adapter versions nor distinguish update propagation from the recomputation required for behavioral recovery. To address these gaps, we introduce CacheReforge, which represents stale KV caches as layerwise mixed-version objects. It combines per-layer adapter anchors, calibrated sensitivity, accumulated drift, and executable restart boundaries to select direct reuse, bounded recomputation, or complete affected-suffix recovery. We distinguish dependency depth from the functional recomputation horizon and use cumulative tail influence to characterize when bounded recovery preserves current-model behavior. We evaluate CacheReforge on Qwen2.5-1.5B and Qwen2.5-7B with continual LoRA updates, including 16K HotpotQA and 2WikiMQA workloads. CacheReforge reduces mean KL divergence by 92.4% relative to stale reuse, while recomputing only 5.44% of layers and reducing cache-maintenance time by 93.2% relative to fresh full prefill. These results show that version-aware recovery preserves model fidelity and most KV caching gains.

Figures & tables

Figure 1: Overview of Cache Recovery under Adapter Evolution.

Figure 2: Overview of CacheReforge workflow.

Figure 3: Adapter evolution and stale-KV mismatch on Qwen2.5-7B. Top: KL divergence between stale reuse and fresh recomputation. Bottom: task-score change from each workload’s pre-update baseline.

Figure 4: Empirical recomputation horizons.

Figure 5: Hidden-state propagation after early-layer updates.

Method	HotpotQA (%) $\uparrow$	2WikiMQA (%) $\uparrow$	KL $\downarrow$	Recomp. (%) $\downarrow$	Maint. (s) $\downarrow$
Fresh full	9.41	9.40	0.0000	100.00	50.26
Exact affected suffix	9.41	9.40	0.0000	50.00	28.75
CacheReforge	9.13	9.16	0.0335	5.44	3.42
Stale reuse	7.95	9.03	0.4385	0.00	0.31

Table 1: Held-out 32-version quality–maintenance-cost comparison on HotpotQA and 2WikiMQA.

Method	KL $\downarrow$	Top-1 (%) $\uparrow$	Recomp. (%) $\downarrow$
CacheReforge	0.0335	94.04	5.44
Drift-only	0.0581	92.22	3.82
Fixed-window	0.2198	88.05	7.14

Table 2: Recovery-policy ablations on the frozen RQ4 trajectories.

Term or symbol	Meaning
Adapter evolution	Successive updates to the same lightweight adapter while the base model remains frozen.
Stale KV cache	$\mathcal{C}^{(t)}$ is generated under $v_{t}$ but retained under $v_{t+\Delta}$ ; the prefix is unchanged, but the cache and current model state are mismatched.
Recovery modes	Direct reuse executes $h=0$ layers; bounded recovery has $0<h<N-r(c)$ ; exact affected-suffix recomputation executes the complete executable suffix, $h=N-r(c)$ ; fresh full recomputation reruns all $N$ layers and defines the reference.
Dependency depth vs. horizon	Dependency depth is the exact downstream propagation extent. $H_{\epsilon}(c)$ is the minimum executable recovery length meeting population tolerance $\epsilon$ ; the RQ2 empirical horizon is the first tested candidate passing both aggregate and per-sample criteria.
$N,\ell,s,\Delta$	Number of layers, generic layer index, updated layer, and adapter version gap.
$r_{\mathrm{KV}}(m,s),r(c)$	Earliest cached layer that can change after updating module $m$ at layer $s$ , and earliest executable restart layer. They can differ for query, output, and MLP updates.

Table S1: Reference terminology and notation. All layer ranges are half-open.

Condition	Suffix depth	D/B/F	Observed horizons
Key–early	28	2/6/1	$\{0,1,8,16,28\}$
Key–late	1	7/0/2	$\{0,1\}$
Value–early	28	3/2/4	$\{0,1,4,28\}$
Query–early	28	6/1/2	$\{0,16,28\}$
Query–middle	14	1/6/2	$\{0,2,4,8,14\}$
MLP–early	28	3/1/5	$\{0,16,28\}$

Table S2: Recovery outcomes by updated module and position. D/B/F denotes direct, bounded, and full-suffix outcomes.

Metric	Overall $\rho$	Within $\rho$
Hidden tail sum	0.873	0.690
Hidden boundary	0.666	0.320
Key tail sum	0.853	0.690
Value tail sum	0.853	0.690
Final suffix-attention input	0.892	0.620

Table S3: Spearman correlation between candidate-window KL and representative propagation metrics.

HotpotQA
Method	Score	KL	Top-1	Recomp.
Fresh full	9.41	0.0000	100.00	100.00
Exact affected suffix	9.41	0.0000	100.00	50.00
Stale reuse	7.95	0.3923	76.56	0.00
CacheReforge	9.13	0.0197	93.75	6.25
Drift-only	9.41	0.0595	89.19	3.12

Table S4: Taskwise held-out RQ4 results (all values except KL are percentages). Each task contains 48 complete 32-version trajectories.

KV-Cache Management

Key-value (KV) cache management reduces and coordinates Transformer attention state to make long-context inference faster and more memory-efficient.

See all 201 papers

PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts
Aug 18, 2026Guotao Yang, Rui Guo, Siwei He +3KV-Cache Reuse LLM Inference Efficiency
Tianjin University, China
ATTUNER: Recomputation-Free Agentic Context KV Reuse
Sep 29, 2026Xinghao Chen, Junnan Dong, Cai Ke +8KV-Cache Reuse LLM Inference Acceleration
EIT-NLP Lab, Eastern Institute of Technology, Ningbo · The Hong Kong Polytechnic University · Tencent Youtu Lab
RaReCache: Bridging the Gap in Cross-Model KV Cache Reuse via Rank disagreement-based Selective Recomputation
Oct 8, 2026Sreetama Sarkar, Saptarshi Mitra, Sitao Huang +2KV-Cache Reuse LLM Inference Acceleration
University of Southern California · University of California, Irvine · Intel Labs

Aug 18, 2026J/K move · Enter open · S save

PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts

Guotao Yang, Rui Guo, Siwei He, Sheng Chen+2

Tianjin University, China

KV-Cache Reuse LLM Inference Efficiency KV-Cache Offloading

Open paper AskPDF arXiv

Long-running LLM agent workflows often revise interior context spans while retaining long suffixes. Although suffix tokens remain unchanged, altered causal histories and rotary positions prevent exact reuse of their offloaded key-value (KV) states. Full suffix recomputation wastes prefill work, while indiscriminate reuse propagates stale states and full-precision restoration adds data movement. We present PatchKV, a profile-guided recovery system for suffix-preserving revisions. PatchKV decomposes adjacent context versions into an exact prefix, an updated span, and an aligned suffix. It predicts an edit-local dirty region using an offline length-conditioned drift model, augments this region with sparse nonlocal blocks selected from stored attention, and block-rounds their union into a fixed repair set. The remaining suffix blocks are restored from CPU memory using frozen per-block precision tags and a fused path for dequantization, RoPE correction, and KV-page placement. Across three models and three long-context question-answering workloads, PatchKV achieves a

2.51

3.85\times

speedup in mean resume time-to-first-token over full suffix recomputation and a

1.26

2.06\times

speedup over CacheBlend, while matching or exceeding CacheBlend's F1 score in six of nine settings and remaining within 1.36 points in the others.

Abstract

KV-Cache Management

Explore similar work

PatchKV: Efficient KV Cache Recovery for Dynamically Edited LLM Contexts

ATTUNER: Recomputation-Free Agentic Context KV Reuse

RaReCache: Bridging the Gap in Cross-Model KV Cache Reuse via Rank disagreement-based Selective Recomputation