Production deployments often use different models to balance cost and quality through cascading, routing, and switching models during long conversations. Context accumulates across turns. Each switch requires the target to repeat prefill over this context. Existing methods transfer full-attention (FA) K/V caches across models. Hybrid linear-attention models also retain recurrent states (REC) and convolution histories (Conv). The interaction between Conv and REC complicates cross-model state transfer. Conv perturbations can change keys and values in recurrent layers and alter REC updates. Conv histories are overwritten, but changes in REC can still affect later outputs. More accurate REC reconstruction need not improve continuation. Building on this, we propose Hybrid State Transfer (HST) for all three state types. It uses structured REC maps and preserves history order during Conv alignment. We calibrate the adapters using the frozen target's outputs during continuation. FA cache map updates are merged offline. Across five benchmarks, Qwen3.5 4B→27B and 9B→27B achieve mean task retentions of 89.88% and 94.11%, respectively. With source caches already available, separate timing on synthetic 16K inputs yields handoff body speedups of 21.66× and 21.91×.
Figures & tables
Figure 1: State restoration after zeroing Conv in native Qwen3.5-9B. Curves show all 32 steps, with two prefix boundaries averaged within each of eight packed blocks before averaging across blocks. Restore both denotes REC+Conv; FA caches are not restored. The dashed line marks restoration after measuring step 4. Both axes are logarithmic; KL is from native outputs to each branch.
Configuration
Initial REC relative MSE ↓
Continuation KL ↓
Base
1.00671
0.64829
REC residual
0.28380
0.69142
Context residual
0.24696
0.67841
Table 1: Initial REC reconstruction error and full-model continuation KL for the auxiliary 4B → 27B configuration on 32 development documents used for candidate selection. Results are averaged over nine conditions per document: three prefix lengths (128, 512, and 2048 tokens) crossed with three continuation branches ( natural , revisit , and topic shift ). All configurations consume the same fixed continuation tokens. Lower reconstruction error need not yield lower continuation KL.
Figure 2: Handoff protocol and body latency definition. Given an available source state at C=N−r , handoff skips target prefill of x1:C . The timed handoff includes state conversion, installation, the REC correction when used, and target suffix processing. The native reference processes the complete prompt. Both timed paths end when the target cache and final hidden state are ready. Source prefill, the vocabulary output projection, and answer generation are excluded. Block widths are schematic.
Figure 3: Complete-state handoff for Qwen3.5-4B → 27B with a state-conditioned REC correction. Offline training calibrates the REC mappings and trains the correction before the final FA-only stage. FA increments are merged into the cache-mapping weights before handoff. At C=N−r , the adapter converts FA K/V caches, REC matrices, and Conv histories. The REC correction produces S+ from S0 once. All target states and position metadata are ready before the target processes xC+1:N .
Figure 4: Body latency comparison between state transfer and native target prefill for 9B → 27B on synthetic contexts with an available source cache. At 16K tokens, transfer reduces latency from 8,823.37 to 402.79 ms, yielding a 21.91× speedup.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Model / direction
Method
ARC-C
HellaSwag
WinoGrande
MMLU
GSM8K
Macro
4B
Native
54.27
73.12
70.24
69.72
89.39
71.35
9B
Native
56.14
78.08
72.85
69.92
93.40
74.08
27B
Native
61.52
83.37
78.77
86.51
96.36
81.31
4B → 27B
Ridge
47.18
70.49
52.88
57.14
47.61
55.06
CacheBridge
46.67
69.33
51.93
53.53
5.53
45.40
CacheLink
55.55
77.02
68.43
64.65
84.15
69.96
Appendix
Table 3: Absolute benchmark accuracy (%). The native target provides the reference for each transfer direction. Macro equally averages tasks.
Compatible configuration
ARC-C
HellaSwag
WinoGrande
MMLU
GSM8K
Avg.
4B → 9B (Ridge)
101.23
96.50
94.15
102.85
90.50
97.05
4B → 9B (Ours)
99.70
97.45
98.16
99.71
94.56
97.92
9B → 4B (Ours)
98.36
98.97
98.36
107.00
97.34
100.01
Appendix
Table 4: Task retention (%) for matching-layout handoffs. Native 9B and native 4B are the respective references for 4B → 9B and 9B → 4B. Avg. equally averages task retentions.
Configuration
FA increment
REC correction
Correct / total
Accuracy (%)
A
Off
Off
832 / 1319
63.08
B
Off
On
878 / 1319
66.57
C
On
Off
1144 / 1319
86.73
D
On
On
1165 / 1319
88.32
Native 27B
—
—
1270 / 1319
96.29
Appendix
Table 5: GSM8K deployment ablation in a fixed auxiliary configuration. Modules are toggled without retraining.
Native-state replacement
Composite KL ↓
None
0.59291
REC only
0.10060
FA only
0.33515
REC and FA
0.01664
Conv only
0.61845
Appendix
Table 6: Native-state replacement on 18 documents. Only the specified components are replaced with native target states for the same prefix. Lower composite KL is better.
4B → 27B
9B → 27B
Input length
Native (ms)
Transfer (ms)
Speedup
Native (ms)
Transfer (ms)
Speedup
64
367.43
132.20
2.78×
365.41
121.46
3.01×
128
384.88
134.32
2.87×
383.93
123.83
3.10×
256
410.29
135.71
3.02×
410.48
126.12
3.25×
512
468.38
139.97
3.35×
462.58
127.43
3.63×
1K
589.90
147.09
4.01×
593.08
136.37
4.35×
Appendix
Table 7: Native target and handoff body times in milliseconds. K denotes 1024 tokens; speedup is native time divided by transfer time. Values are rounded to two decimal places.
Configuration
Additional FA parameters (M)
Updates
Stage time (h)
Main 4B → 27B
0.655
768
2.92
Main 9B → 27B
0.655
768
3.36
Appendix
Table 8: Final FA-stage parameters, updates, and elapsed times for the main systems. Each stage uses 384 training units. Times cover final FA updates only.
Step
REC+Conv
REC+Conv+FA
18
0.00703398
0.00000000
19
0.01039322
0.00000000
20
0.00508434
0.00000000
Appendix
Table 9: Output KL after oracle restoration in native Qwen3.5-9B. Both columns use the same eight blocks and two cuts per block.
Can one language model hand its live memory to another without the receiver rereading the context? We demonstrate useful persistent hybrid-state transfer across one architecture-matched Qwen3.5 4B-to-9B sibling pair. To our knowledge, this is the first demonstrated cross-model handoff of persistent recurrent inference state between differently sized hybrid language models without target prefix replay. Translated attention KV alone leaves a large gap; adding the Gated DeltaNet (GDN) persistent-state package lowers teacher-forced negative log-likelihood (NLL), the average next-token log-loss, by 0.747 nats/token (95% paired document bootstrap CI [0.6921, 0.8047]), improving all 64 PG19 documents. Direct recurrent and convolution reuse outperforms the tested learned GDN maps, consistent with partial functional compatibility of persistent-state coordinates. A fresh component factorial selects translated KV with direct recurrent and convolution state. An additional 434,176-parameter correction improves that base on 64 fresh web documents: continuation loss is 0.076 nats/token above native 9B (excess NLL), Jensen-Shannon (JS) divergence is 0.022, and native context recovery (NCR) is 0.918. Corrected 9B significantly beats continued 4B inference while processing zero historical prefix tokens. Evidence covers one direction, one geometry-matched Base-model pair, and 4K teacher-forced continuation; the near-native gate failed, the 16K branch was not run, and free-generation equivalence and a general state interface remain unproven.
Hybrid language models combine attention with a fixed-size recurrent state, but the role of each channel remains unclear. We introduce two cache-level interventions. Split-prefill keeps only the KV cache or only the recurrent state from a prefilled context, then generates an answer. State-swap pairs the KV cache from one context with the recurrent state from another in a single forward pass. On Qwen3.5 and Falcon-H1, the two channels split sharply by function. Exact retrieval survives only through attention (64-98% of full accuracy) and collapses to zero through recurrence. Output language and persona reverse the pattern: both survive recurrence (70-80% and 3-5x) while KV-only drops to ~1% language accuracy. State-swap confirms this causally: the answer takes its value from the KV side and its language from the recurrent side. Recurrent-only generation also accepts words that were never in the context but share meaning or parts with seen items. Attention provides a lookup over what was said; the recurrent state shapes how the model says it next.
Kirill Afendulev, Alexey Dontsov, Elena Tutubalina +1
AIRI, Moscow, Russia · YSDA, Moscow, Russia · HSE University, Moscow, Russia
Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch. We propose cross-model KV cache transfer, where the receiver reuses the source's KV cache, skipping prefill. We find that cross-model KV has substantial linear structure across matched-KV pairs, where source and target share KV head count and per-head dimension. On Qwen3 14B->32B, one source layer explains 56% of variance in the target's keys and 32% in values, rising to 79% and 65% with multiple source layers. Building on this, we design a closed-form ridge mapper that operates per head and proceeds in three steps. First, for each target layer we select the top-k most predictive source layers and concatenate their KV as input. Second, we strip RoPE from the keys before mapping, so the fit is position-free and reusable across context lengths. Third, we fit ridge regression on a small calibration set of 500 FineWeb-Edu sequences of 1,024 tokens each. Surprisingly, across six pairs in three families, this linear mapper retains 73-98% of the receiver's standalone-prefill accuracy on four pairs, while two degrade sharply. A nonlinear MLP recovers up to +37 pp HellaSwag retention on the failures. The mapper runs 2.7-25x faster than re-prefill and remains stable across multi-turn handoff, making cross-model KV cache transfer practical.