LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay

Simon P. Villani

cs.CLSep 7, 2026

LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay

Authors: Simon P. Villani

Abstract

Can one language model hand its live memory to another without the receiver rereading the context? We demonstrate useful persistent hybrid-state transfer across one architecture-matched Qwen3.5 4B-to-9B sibling pair. To our knowledge, this is the first demonstrated cross-model handoff of persistent recurrent inference state between differently sized hybrid language models without target prefix replay. Translated attention KV alone leaves a large gap; adding the Gated DeltaNet (GDN) persistent-state package lowers teacher-forced negative log-likelihood (NLL), the average next-token log-loss, by 0.747 nats/token (95% paired document bootstrap CI [0.6921, 0.8047]), improving all 64 PG19 documents. Direct recurrent and convolution reuse outperforms the tested learned GDN maps, consistent with partial functional compatibility of persistent-state coordinates. A fresh component factorial selects translated KV with direct recurrent and convolution state. An additional 434,176-parameter correction improves that base on 64 fresh web documents: continuation loss is 0.076 nats/token above native 9B (excess NLL), Jensen-Shannon (JS) divergence is 0.022, and native context recovery (NCR) is 0.918. Corrected 9B significantly beats continued 4B inference while processing zero historical prefix tokens. Evidence covers one direction, one geometry-matched Base-model pair, and 4K teacher-forced continuation; the near-native gate failed, the 16K branch was not run, and free-generation equivalence and a general state interface remain unproven.

Figures & tables

Figure 1: What crosses the model boundary. The selected handoff translates attention KV, directly reuses GDN recurrent matrices and convolution history, and applies a small additional residual correction before installing target-compatible persistent state into the 9B runtime. The receiver processes a real bridge token and subsequent continuation, with zero historical prefix tokens . Matched geometry permits copying despite different hidden widths and weights; usefulness is tested behaviorally. This schematic does not measure latency.

Figure 2: Beyond KV, then a corrected full-state handoff. (A) Adding the full translated GDN persistent-state package to fixed translated KV improves all 64 paired PG19 documents, lowering NLL by 0.7473 nats/token (95% CI [0.6921, 0.8047]). The mean per-document reduction of KV-only excess NLL is 79.7%; recurrent matrices, convolution history, and initialization semantics are not isolated in this contrast. (B) On 64 different FineWeb-Edu test documents, the corrected handoff beats continued 4B inference by 0.0521 nats/token (95% CI [0.0185, 0.0843]). The corrected handoff recovers 91.8% of native 9B’s prefix-derived NLL improvement relative to empty 9B; this is context-benefit recovery, not accuracy. Points are document means; intervals concern paired differences. Both experiments score 64 teacher-forced targets after a 4K prefix. Panel scales and corpora differ.

Property	4B	9B
Parameters (billions)	4.206	8.954
Hidden width	2560	4096
Language layers	32	32
GDN / attention layers	24 / 8	24 / 8
Recurrent state / layer	$32\!\times\!128\!\times\!128$	$32\!\times\!128\!\times\!128$
Convolution state / layer	$8192\!\times\!4$	$8192\!\times\!4$

Table 1: Runtime-verified correspondence. State shapes omit batch and are per GDN layer; each attention K or V tensor is

4\times L\times 256

. Persistent geometry matches even though model size and hidden width differ.

Metric	Meaning	Reference
NLL	Average log-loss on observed continuation	Lower
Excess NLL	Handoff NLL minus native 9B NLL	0
NCR	Fraction of native 9B context benefit recovered	1
JS divergence	Difference from native 9B output distribution	0
Top-1 agreement	Same highest-probability next token as native 9B	Higher

Table 2: Metrics at a glance. Reference values describe loss or agreement with native 9B, not task accuracy.

Figure 3: 64 of 64 documents improve. Each point pairs one PG19 document’s excess NLL under KV-only and full translated state; every point lies below equality. Adding the GDN package improves NLL by 0.7473 nats/token (95% paired document bootstrap CI [0.6921, 0.8047]). The diamond marks paired means. Prefixes contain 4,096 tokens and each document contributes 64 teacher-forced targets. This is a document-level effect, not 4,096 independent token replicates.

Condition	NLL	$\Delta$ NLL	Top-1
Native 9B	2.1609	0.0000	1.000
Source 4B	2.3074	0.1465	–
Empty 9B	3.2457	1.0847	0.581
KV only	3.1153	0.9544	0.600
KV + direct GDN	2.2913	0.1304	0.821
Full translated	2.3679	0.2070	0.783

Table 3: Experiment 1 test fidelity on 64 PG19 documents. NLLs are document means in nats/token; top-1 is agreement with native 9B. Source agreement was not recorded in this table. Full precision is retained in Appendix E .

Figure 4: Translate KV; directly reuse recurrent memory. Experiment 2’s factorial uses 32 fresh FineWeb-Edu validation documents, 4K prefixes, and 64 teacher-forced targets. (A) D/T denotes direct/translated

(K,R,C)

state; filled points translate KV. TDT has the lowest numerical loss, but the frozen 0.01-nat tie rule selects TDD (diamond), which uses fewer translations. (B) Main effects average paired differences over the other factors; bars are 95% document bootstrap intervals. Recurrent translation worsens NLL, whereas convolution translation has an uncertain small effect.

Condition	NLL	$\Delta$ NLL	JS	NCR	Top-1
Native 9B	1.913	0.000	0.000	1.000	1.000
Source 4B	2.042	0.129	0.038	0.862	0.834
Empty 9B	2.842	0.929	0.157	0.000	0.626
TDD / base	2.018	0.105	0.029	0.886	0.846
TTT	2.047	0.134	0.036	0.855	0.832
Joint corrected	1.989	0.076	0.022	0.918	0.864

Table 4: Experiment 2 test fidelity on 64 fresh FineWeb-Edu documents at 4K, with 64 teacher-forced targets each. Corrected 9B improves on both its selected base and continued 4B; wrong-donor state removes the advantage. NLL and excess NLL are nats/token, JS compares distributions with native 9B, and top-1 measures prediction agreement. NCR is context-benefit recovery (Eq. 8 ). Six-decimal values and TQR appear in Appendix E .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Rank	$\lambda_{\rm id}$	Epoch	Val. KL
2	0	3	0.171111
2	0.0001	3	0.169789
2	0.001	3	0.170405
2	0.01	3	0.170945
4	0	3	0.162947
4	0.0001	3	0.163098

Appendix

Table 5: Frozen E002 correction selection grid. Validation KL is mean full-vocabulary native-to-handoff KL over nine output positions on 32 non-LOCKED 1K documents. These are selection scores, not LOCKED results.

Component	Parameters	Median norm
KV (rank 4)	16,384	0.027093
Recurrent (rank 4)	24,576	0.031916
Conv. scale / bias	393,216	0.015320
Total / pooled	434,176	0.026432

Appendix

Table 6: Frozen E002 correction inventory. Median norm is the residual/base Frobenius norm over LOCKED document–layer records, pooled separately by component or over all records. Rank 4 applies to KV and recurrent residuals; convolution uses channel scale/bias. Counts exclude fixed bases and the existing KV translator.

Condition	NLL	$\Delta$ NLL	Top-1
Native 9B	2.160916	0.000000	1.000000
Source 4B	2.307412	0.146496	–
Empty 9B	3.245654	1.084738	0.581299
KV only	3.115271	0.954355	0.600098
KV + direct GDN	2.291314	0.130398	0.821289
Full translated	2.367948	0.207032	0.783203

Appendix

Table 7: E001 LOCKED results: 64 PG19 documents, 4K prefix, 64 teacher-forced targets. NLL and excess NLL are nats/token; top-1 is native-target prediction agreement.

Condition	NLL	$\Delta$ NLL	NCR	TQR	Top-1	JS
Native 9B	1.913032	0.000000	1.000000	1.000000	1.000000	0.000000
Source 4B	2.041544	0.128512	0.861597	0.000000	0.833984	0.038052
Empty 9B	2.841564	0.928532	0.000000	-6.225273	0.626465	0.156773
TDD / base	2.018422	0.105390	0.886498	0.179921	0.846436	0.028583
TTT	2.047468	0.134437	0.855216	-0.046104	0.832031	0.036172
Joint corrected	1.989399	0.076367	0.917755	0.405755	0.864258	0.022067

Appendix

Table 8: E002 LOCKED results: 64 fresh FineWeb-Edu documents, 4K prefix, 64 teacher-forced targets. All entries are document means or defined ratios of document means. NLL and excess NLL use nats/token. NCR, TQR, top-1 agreement, and JS have distinct meanings; native/empty NCR and native/source TQR are fixed by definition.

Figure 5: E002’s complete test comparison. Markers are means over 64 paired documents. The left axis includes empty and shuffled controls; the right expands the scale and includes TTT. Corrected excess NLL is 0.076367. Base-minus-corrected improvement is 0.029022 nats/token (95% paired document bootstrap CI [0.022828, 0.035413]); plotted means are not independent token-level replicates.

Cell	Validation	LOCKED
DDD	0.217430	0.165163
DDT	0.217473	0.166589
DTD	0.231225	0.178138
DTT	0.231250	0.180053
TDD	0.133377	0.105390
TDT	0.132226	0.105677

Appendix

Table 9: Every E002 factorial cell’s excess NLL, in nats/token. Validation has 32 fresh FineWeb-Edu documents; LOCKED has 64 different documents. Both use 4K prefixes. Base selection uses validation alone.

Contrast	Validation	95% CI	LOCKED	95% CI
KV	-0.070463	[-0.100867, -0.040946]	-0.052809	[-0.069557, -0.037096]
Recurrent	0.027973	[0.010360, 0.047548]	0.020753	[0.007905, 0.034039]
Convolution	-0.000429	[-0.002647, 0.001774]	0.001216	[-0.000055, 0.002580]
$K\times R$	0.028373	[0.008556, 0.057582]	0.015067	[0.008212, 0.022371]
$K\times C$	-0.000927	[-0.002873, 0.001039]	-0.000910	[-0.002206, 0.000378]
$R\times C$	0.000250	[-0.002447, 0.003218]	0.000718	[-0.001490, 0.003543]

Appendix

Table 10: E002 orthogonal within-document factorial contrasts, in nats/token, with 95% paired document bootstrap intervals. Main effects are translated minus direct, averaged over the other two components. Pairwise material replication requires matching signs and absolute LOCKED estimate at least 0.02; no pairwise contrast satisfies both requirements.

Figure 6: Secondary recurrent-state convergence under shared teacher-forced inputs. Each marker averages layer/head state errors over 64 documents within the indicated experiment. The horizontal axis is logarithmic with only the recorded checkpoints shown. E001 and E002 use different LOCKED corpora and are not paired with one another. E002’s 256-input state measurement is not a 16K-context test or a free-generation evaluation.

Table 17

Hybrid Language Models

See all 14 papers

What Attention Recalls and Recurrence Controls in Hybrid Language Models
Sep 3, 2026Kirill Afendulev, Alexey Dontsov, Elena Tutubalina +1Hybrid Attention Recurrent Transformers
AIRI, Moscow, Russia · YSDA, Moscow, Russia · HSE University, Moscow, Russia
A Universal Context-Reuse Layer for Cross-Model KV Sharing
Aug 31, 2026Yi Li, Dongming Jiang, Yi Zhao +1LLM Inference KV Caching
University of Texas at Dallas
The Key Handoff: Retrieval in Hybrid Language Models
Sep 26, 2026Kaan Kale, Oguzhan Baser, Sriram VishwanathHybrid Language Models Hybrid Transformer-State Space Model Architectures
Georgia Institute of Technology, USA · University of Texas at Austin, USA

Sep 3, 2026J/K move · Enter open · S save

What Attention Recalls and Recurrence Controls in Hybrid Language Models

Kirill Afendulev, Alexey Dontsov, Elena Tutubalina, Anton Korznikov

AIRI, Moscow, Russia · YSDA, Moscow, Russia · HSE University, Moscow, Russia

Hybrid Attention Recurrent Transformers In-Context Retrieval Causal Interventions in Language Models

Open paper AskPDF arXiv

Hybrid language models combine attention with a fixed-size recurrent state, but the role of each channel remains unclear. We introduce two cache-level interventions. Split-prefill keeps only the KV cache or only the recurrent state from a prefilled context, then generates an answer. State-swap pairs the KV cache from one context with the recurrent state from another in a single forward pass. On Qwen3.5 and Falcon-H1, the two channels split sharply by function. Exact retrieval survives only through attention (64-98% of full accuracy) and collapses to zero through recurrence. Output language and persona reverse the pattern: both survive recurrence (70-80% and 3-5x) while KV-only drops to ~1% language accuracy. State-swap confirms this causally: the answer takes its value from the KV side and its language from the recurrent side. Recurrent-only generation also accepts words that were never in the context but share meaning or parts with seen items. Attention provides a lookup over what was said; the recurrent state shapes how the model says it next.

Abstract

Hybrid Language Models

Explore similar work

What Attention Recalls and Recurrence Controls in Hybrid Language Models

A Universal Context-Reuse Layer for Cross-Model KV Sharing

The Key Handoff: Retrieval in Hybrid Language Models