Beyond K/V Caches: Cross-Model State Handoff in Hybrid Linear-Attention LLMs

Jiacheng Cai; Tao Zhang; Siqi Liang; Yun Dai; Ziqian Zeng

cs.CLNov 5, 2026

Beyond K/V Caches: Cross-Model State Handoff in Hybrid Linear-Attention LLMs

Authors: Jiacheng Cai, Tao Zhang, Siqi Liang, Yun Dai, Ziqian Zeng

Organizations: South China University of Technology, China

Abstract

Production deployments often use different models to balance cost and quality through cascading, routing, and switching models during long conversations. Context accumulates across turns. Each switch requires the target to repeat prefill over this context. Existing methods transfer full-attention (FA) K/V caches across models. Hybrid linear-attention models also retain recurrent states (REC) and convolution histories (Conv). The interaction between Conv and REC complicates cross-model state transfer. Conv perturbations can change keys and values in recurrent layers and alter REC updates. Conv histories are overwritten, but changes in REC can still affect later outputs. More accurate REC reconstruction need not improve continuation. Building on this, we propose Hybrid State Transfer (HST) for all three state types. It uses structured REC maps and preserves history order during Conv alignment. We calibrate the adapters using the frozen target's outputs during continuation. FA cache map updates are merged offline. Across five benchmarks, Qwen3.5 4B $\rightarrow$ 27B and 9B $\rightarrow$ 27B achieve mean task retentions of 89.88% and 94.11%, respectively. With source caches already available, separate timing on synthetic 16K inputs yields handoff body speedups of $21.66\times$ and $21.91\times$ .

Figures & tables

Figure 1: State restoration after zeroing Conv in native Qwen3.5-9B. Curves show all 32 steps, with two prefix boundaries averaged within each of eight packed blocks before averaging across blocks. Restore both denotes REC+Conv; FA caches are not restored. The dashed line marks restoration after measuring step 4. Both axes are logarithmic; KL is from native outputs to each branch.

Configuration	Initial REC relative MSE $\downarrow$	Continuation KL $\downarrow$
Base	1.00671	0.64829
REC residual	0.28380	0.69142
Context residual	0.24696	0.67841

Table 1: Initial REC reconstruction error and full-model continuation KL for the auxiliary 4B

\rightarrow

27B configuration on 32 development documents used for candidate selection. Results are averaged over nine conditions per document: three prefix lengths (128, 512, and 2048 tokens) crossed with three continuation branches ( natural , revisit , and topic shift ). All configurations consume the same fixed continuation tokens. Lower reconstruction error need not yield lower continuation KL.

Figure 2: Handoff protocol and body latency definition. Given an available source state at

C=N-r

, handoff skips target prefill of

x_{1:C}

. The timed handoff includes state conversion, installation, the REC correction when used, and target suffix processing. The native reference processes the complete prompt. Both timed paths end when the target cache and final hidden state are ready. Source prefill, the vocabulary output projection, and answer generation are excluded. Block widths are schematic.

Figure 3: Complete-state handoff for Qwen3.5-4B

\rightarrow

27B with a state-conditioned REC correction. Offline training calibrates the REC mappings and trains the correction before the final FA-only stage. FA increments are merged into the cache-mapping weights before handoff. At

C=N-r

, the adapter converts FA K/V caches, REC matrices, and Conv histories. The REC correction produces

S^{+}

from

S^{0}

once. All target states and position metadata are ready before the target processes

x_{C+1:N}

Figure 4: Body latency comparison between state transfer and native target prefill for 9B

\rightarrow

27B on synthetic contexts with an available source cache. At 16K tokens, transfer reduces latency from 8,823.37 to 402.79 ms, yielding a

21.91\times

speedup.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model / direction	Method	ARC-C	HellaSwag	WinoGrande	MMLU	GSM8K	Macro
4B	Native	54.27	73.12	70.24	69.72	89.39	71.35
9B	Native	56.14	78.08	72.85	69.92	93.40	74.08
27B	Native	61.52	83.37	78.77	86.51	96.36	81.31
4B $\rightarrow$ 27B	Ridge	47.18	70.49	52.88	57.14	47.61	55.06
	CacheBridge	46.67	69.33	51.93	53.53	5.53	45.40
	CacheLink	55.55	77.02	68.43	64.65	84.15	69.96

Appendix

Table 3: Absolute benchmark accuracy (%). The native target provides the reference for each transfer direction. Macro equally averages tasks.

Compatible configuration	ARC-C	HellaSwag	WinoGrande	MMLU	GSM8K	Avg.
4B $\rightarrow$ 9B (Ridge)	101.23	96.50	94.15	102.85	90.50	97.05
4B $\rightarrow$ 9B (Ours)	99.70	97.45	98.16	99.71	94.56	97.92
9B $\rightarrow$ 4B (Ours)	98.36	98.97	98.36	107.00	97.34	100.01

Appendix

Table 4: Task retention (%) for matching-layout handoffs. Native 9B and native 4B are the respective references for 4B

\rightarrow

9B and 9B

\rightarrow

4B. Avg. equally averages task retentions.

Configuration	FA increment	REC correction	Correct / total	Accuracy (%)
A	Off	Off	832 / 1319	63.08
B	Off	On	878 / 1319	66.57
C	On	Off	1144 / 1319	86.73
D	On	On	1165 / 1319	88.32
Native 27B	—	—	1270 / 1319	96.29

Appendix

Table 5: GSM8K deployment ablation in a fixed auxiliary configuration. Modules are toggled without retraining.

Native-state replacement	Composite KL $\downarrow$
None	0.59291
REC only	0.10060
FA only	0.33515
REC and FA	0.01664
Conv only	0.61845

Appendix

Table 6: Native-state replacement on 18 documents. Only the specified components are replaced with native target states for the same prefix. Lower composite KL is better.

Input length	Native (ms)	Transfer (ms)	Speedup	Native (ms)	Transfer (ms)	Speedup
	4B $\rightarrow$ 27B			9B $\rightarrow$ 27B
64	367.43	132.20	$2.78\times$	365.41	121.46	$3.01\times$
128	384.88	134.32	$2.87\times$	383.93	123.83	$3.10\times$
256	410.29	135.71	$3.02\times$	410.48	126.12	$3.25\times$
512	468.38	139.97	$3.35\times$	462.58	127.43	$3.63\times$
1K	589.90	147.09	$4.01\times$	593.08	136.37	$4.35\times$

Appendix

Table 7: Native target and handoff body times in milliseconds. K denotes 1024 tokens; speedup is native time divided by transfer time. Values are rounded to two decimal places.

Configuration	Additional FA parameters (M)	Updates	Stage time (h)
Main 4B $\rightarrow$ 27B	0.655	768	2.92
Main 9B $\rightarrow$ 27B	0.655	768	3.36

Appendix

Table 8: Final FA-stage parameters, updates, and elapsed times for the main systems. Each stage uses 384 training units. Times cover final FA updates only.

Step	REC+Conv	REC+Conv+FA
18	0.00703398	0.00000000
19	0.01039322	0.00000000
20	0.00508434	0.00000000

Appendix

Table 9: Output KL after oracle restoration in native Qwen3.5-9B. Both columns use the same eight blocks and two cuts per block.

KV-Cache Transfer

See all 23 papers

LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay
Sep 7, 2026Simon P. VillaniHybrid Language Models Language Modeling
What Attention Recalls and Recurrence Controls in Hybrid Language Models
Sep 3, 2026Kirill Afendulev, Alexey Dontsov, Elena Tutubalina +1Hybrid Attention Recurrent Transformers
AIRI, Moscow, Russia · YSDA, Moscow, Russia · HSE University, Moscow, Russia
Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse
Aug 4, 2026Taekyung Heo, Rasoul Shafipour, Ritchie Zhao +6KV Caching LLM Inference Acceleration
NVIDIA

Sep 7, 2026J/K move · Enter open · S save

LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay

Simon P. Villani

Hybrid Language Models Language Modeling Persistent Memory for Language Models

Open paper AskPDF arXiv

Can one language model hand its live memory to another without the receiver rereading the context? We demonstrate useful persistent hybrid-state transfer across one architecture-matched Qwen3.5 4B-to-9B sibling pair. To our knowledge, this is the first demonstrated cross-model handoff of persistent recurrent inference state between differently sized hybrid language models without target prefix replay. Translated attention KV alone leaves a large gap; adding the Gated DeltaNet (GDN) persistent-state package lowers teacher-forced negative log-likelihood (NLL), the average next-token log-loss, by 0.747 nats/token (95% paired document bootstrap CI [0.6921, 0.8047]), improving all 64 PG19 documents. Direct recurrent and convolution reuse outperforms the tested learned GDN maps, consistent with partial functional compatibility of persistent-state coordinates. A fresh component factorial selects translated KV with direct recurrent and convolution state. An additional 434,176-parameter correction improves that base on 64 fresh web documents: continuation loss is 0.076 nats/token above native 9B (excess NLL), Jensen-Shannon (JS) divergence is 0.022, and native context recovery (NCR) is 0.918. Corrected 9B significantly beats continued 4B inference while processing zero historical prefix tokens. Evidence covers one direction, one geometry-matched Base-model pair, and 4K teacher-forced continuation; the near-native gate failed, the 16K branch was not run, and free-generation equivalence and a general state interface remain unproven.

Abstract

KV-Cache Transfer

Explore similar work

LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay

What Attention Recalls and Recurrence Controls in Hybrid Language Models

Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse