MentorPulse: Refreshing Cross-Model Latent Guidance for Long-Form Generation

Ziwu Liu; Guozhong Li; Chen Qiu; Weiyang Kong; Panos Kalnis

cs.CLAug 21, 2026

MentorPulse: Refreshing Cross-Model Latent Guidance for Long-Form Generation

Authors: Ziwu Liu, Guozhong Li, Chen Qiu, Weiyang Kong, Panos Kalnis

Organizations: King Abdullah University of Science and Technology (KAUST)

Abstract

Cross-model latent guidance lets a frozen large mentor encode an input once and a frozen small student generate from the resulting signal. Existing methods keep this signal fixed, assuming it stays useful as the output grows; we show this fails in long-form generation. On multi-turn instruction following, static guidance pushes a 4B student's constraint satisfaction 2.5 points below its no-guidance baseline; a training-free refresh every 16 tokens changes only the memory content and restores a 2.0-point gain over that baseline. We propose MentorPulse to keep guidance fresh at practical cost: it compresses mentor states into a capped slot memory, incrementally processes newly generated tokens, and updates the memory that the student reads through gated cross-attention without resetting the student's KV cache. Windowed Refresh Training exposes the bridge to prefix-conditioned memory. Across thirteen datasets, MentorPulse closes 52.2% of the mentor-student gap on macro average, outperforming C2C, T2T, and equal-budget LoRA, with the largest gains on long outputs. It performs best on all eleven mentor-student pairs from three model families, with margins that narrow as the capability gap grows, and a lightweight read-pattern check predicts the gain before deployment. Measured costs identify refresh intervals that dominate text guidance on long outputs.

Figures & tables

Figure 1: One-shot guidance fixes the memory before decoding and quality decays over long generation (top); MentorPulse refreshes it from the generated prefix every $R$ tokens (bottom).

Figure 2: Staleness diagnosis on Multi-IF. (a) Effect: constraint satisfaction (fraction of verifiable constraints satisfied) by generation segment. (b) Mechanism: probe read accuracy for upcoming-window properties; the shaded wedge is

\mathrm{StaleGap}(t)

Figure 3: MentorPulse data flow. The frozen mentor (blue) prefills, the SMC compresses its states into slot guidance, and the frozen student (orange) reads the slots through the gated bridge; every $R$ tokens an incremental prefill refreshes the memory in place (steps ①–⑦ in the text). Purple components are trained; saturation encodes freshness.

Figure 4: Results on the thirteen datasets for A2; MP uses

R{=}16

, and rT2T refreshes the T2T guidance text on the same schedule (Appendix C.2 ). Dotted/dashed lines are the M-only / S-only anchors; error bars are standard deviations over three seeds; each y-axis is clipped to its anchor range.

Figure 5: The thirteen datasets in one length coordinate system; color is each panel’s recovery rate, clipped to $[-20,100]$ , red marking harm. Coordinates are spread order-preservingly for readability (distances not to scale); point positions are identical in both panels.

#	Ablation	G	K	A	J	$\Delta$ Rec.
1	Full system ( $R{=}16$ )	88.9	82.9	36.7	81.3	—
Refresh mechanism
2	No refresh ( $R\to\infty$ )	79.4	77.6	36.7	81.3	$-41.9$
3	Refresh w/o WRT	80.6	79.8	36.7	81.3	$-32.3$
4	Full re-prefill	88.6	83.0	36.7	81.3	$-0.6$
Memory content

Table 1: Ablations on A2 at

R{=}16

, on two long-output sets (G, K) and two short-answer controls (A, J);

\Delta

Rec. is the change in mean recovery over the four sets relative to row 1. Rows 4 and 11 should stay close to row 1, whereas row 7 must reproduce S-only ; standard deviations in Appendix F .

Pair	$N_{M}/N_{S}$	T2T	C2C	LoRA	MP
A1 (27B $\to$ 9B)	3.0	25.3	9.2	16.3	54.0
A2 (27B $\to$ 4B)	6.8	26.9	10.9	17.5	52.2
A3 (27B $\to$ 2B)	13.5	18.9	8.7	11.8	35.7
B1 (31B $\to$ 12B)	2.6	18.8	7.1	16.9	42.1
C1 (32B $\to$ 8B)	4.0	25.5	11.9	14.9	54.6
C2 (32B $\to$ 4B)	8.2	22.4	11.4	16.0	45.8

Table 2: Macro-average recovery (%) over the thirteen datasets by mentor group; best per row in bold.

N_{M}/N_{S}

is the parameter ratio; MP uses

R{=}16

; averages follow Section 5.1 ; standard deviations in Appendix F ; model names in Figure 6 .

Figure 6: Read-distribution variance

V_{64}

against MP recovery (values as in Table 2 ); marker and color encode the mentor group. Only the rank correlation is reported.

Figure 7: Refresh-interval sweep at inference. (a) Macro-average recovery for all eleven pairs (exclusion rule of Section 5.1 ); the dotted line marks

R{=}16

. (b) A2 with its cross-dataset band, and change relative to

R{=}1

in points.

Figure 8: Cost–quality dominance against T2T on A2. Purple: MP dominates; hatched: neither. Cell numbers give the per-request saving

\mathrm{Cost}_{\mathrm{T2T}}-\mathrm{Cost}_{MP}

10^{-3}

dollars; the dotted line marks

R{=}16

. rT2T is omitted, cost-dominated at every evaluated

R

(Table 27 ).

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Code	Dataset	Metric	Eval. subset	Long-gen. suite
A	MMLU-Pro	accuracy	2,000 (stratified)	—
B	GPQA	accuracy (diamond)	198	—
C	AGIEval-MCQ	accuracy	2,000 (stratified)	—
D	MATH-500	exact-match accuracy	500	✓
E	OlympiadBench	exact-match accuracy (OE-TO-maths-en)	675	✓
F	LiveCodeBench	pass@1	400 (post 2025-07)	—

Appendix

Table 3: The thirteen evaluation datasets. Letter codes are fixed throughout the paper; the last column marks the long-generation suite (inclusion rule in the text; length profiles in Table 4 ).

Code	Dataset	Input med.	Output med.	$L_{g}$ med.
A	MMLU-Pro	410	6	130
B	GPQA	540	92	—
C	AGIEval-MCQ	320	5	—
D	MATH-500	140	430	—
E	OlympiadBench	280	570	—
F	LiveCodeBench	470	520	—

Appendix

Table 4: Length profiles (median token counts). Outputs are measured from M-only generations; the tokenizer is Qwen3.5’s (cross-family deviation

{<}4\%

Condition	Definition	Role	Mentor decodes
S-only	student alone, no bridge, no memory	floor	—
M-only	mentor answers alone	ceiling	yes
T2T	one-shot mentor guidance text ( $L_{g}\leq 384$ ) prepended to the student prompt	text-interface route	yes (once)
rT2T@ $R$	T2T guide refreshed every $R$ tokens; replace-style rebuild, delimiter-stripped	text $\times$ refresh cell (Appendix C.2 )	yes (every refresh)
C2C	mentor layer-wise KV fused into the student cache once before decoding	static-latent route	no
LoRA	equal-budget low-rank finetuning, no memory	parameter route	—

Appendix

Table 5: All experimental conditions. “Mentor decodes” refers to inference time only; offline label generation is excluded. OneShot and the ablation row MP

-

refresh are the same condition under two names (static bridge,

R{\to}\infty

Property	L2S	C2C	LGR	SpecDec	MP
Mentor frozen	✓	✓	✗	✓	✓
Explicit state-count cap	✗	✗	✓	—	✓
Prefix-conditioned refresh	✗	✗	✗	—	✓
Matches mentor distribution	✗	✗	✗	✓	✗

Appendix

Table 6: Property comparison with the closest cross-model interfaces and speculative decoding. Refresh means updating transferred guidance from the generated prefix; dashes mark properties that do not apply.

Item	Value
Long-target prompts / kept labels	26,997 / 20,408
Quality-gate filter rate	5–15% per gate, 24.4% total
Mixed training rows (fuse_v3)	51,908
Long / short-answer mix	20,408 / 31,500 (39% / 61%)
Mentor state store (27B, bf16)	${\sim}1.9$ TB
Max label length	768 tokens

Appendix

Table 7: Training pipeline and hyperparameter summary.

Code	Dataset	M-only	S-only	T2T	C2C	LoRA	MP@16
A	MMLU-Pro	49.5 $\pm$ 0.1	24.1 $\pm$ 0.2	31.6 $\pm$ 0.7	37.0 $\pm$ 0.1	28.9 $\pm$ 0.2	36.7 $\pm$ 0.8
B	GPQA	47.9 $\pm$ 0.2	20.2 $\pm$ 0.2	28.3 $\pm$ 0.8	32.0 $\pm$ 0.8	25.8 $\pm$ 0.7	34.7 $\pm$ 0.4
C	AGIEval-MCQ	61.1 $\pm$ 0.1	40.4 $\pm$ 0.0	46.2 $\pm$ 0.7	49.8 $\pm$ 0.2	44.1 $\pm$ 0.9	51.0 $\pm$ 0.4
D	MATH-500	64.5 $\pm$ 0.1	56.7 $\pm$ 0.1	59.4 $\pm$ 0.4	59.3 $\pm$ 0.3	57.7 $\pm$ 0.4	60.2 $\pm$ 0.3
E	OlympiadBench	26.5 $\pm$ 0.1	20.2 $\pm$ 0.1	21.9 $\pm$ 0.6	22.0 $\pm$ 0.6	21.2 $\pm$ 0.7	23.7 $\pm$ 1.1
F	LiveCodeBench	61.3 $\pm$ 0.2	36.3 $\pm$ 0.2	38.0 $\pm$ 1.2	36.8 $\pm$ 1.4	37.6 $\pm$ 0.5	37.0 $\pm$ 1.2

Appendix

Table 8: Full results for the main pair A2 (Qwen3.5-27B

\to

Qwen3.5-4B), mean

\pm

sd over three seeds; MP@16 throughout. The macro-recovery row uses the definition of Appendix A.4 .

#	Condition	G	K	A	J	Runs
1	MP (full system, $R{=}16$ )	88.9 $\pm$ 0.4	82.9 $\pm$ 1.3	36.7 $\pm$ 0.8	81.3 $\pm$ 0.6	reused ( $=$ MP@16, Tab. 8 )
Refresh mechanism
2	MP $-$ refresh ( $R\to\infty$ )	79.4 $\pm$ 0.2	77.6 $\pm$ 0.9	36.7 $\pm$ 0.8	81.3 $\pm$ 0.6	retrained
3	MP $-$ WRT (zero-training swap)	80.6 $\pm$ 0.5	79.8 $\pm$ 0.9	36.7 $\pm$ 0.8	81.3 $\pm$ 0.6	inference only
4	MP $-$ full (full re-prefill)	88.6 $\pm$ 0.2	83.0 $\pm$ 0.9	36.7 $\pm$ 0.8	81.3 $\pm$ 0.6	inference only
Memory content

Appendix

Table 9: Raw scores with standard deviations for the twelve ablation rows of Table 1 (A2,

R{=}16

; two long-output sets G, K and two short-answer controls A, J). “Reused” rows are copied from their registered source, not re-evaluated.

	M-only	S-only	T2T	C2C	LoRA	MP@16
A1 : Qwen3.5-27B $\to$ Qwen3.5-9B ( $N_{M}/N_{S}=3.0$ )
A	49.5 $\pm$ 0.1	27.4 $\pm$ 0.1	34.7 $\pm$ 0.6	39.0 $\pm$ 0.3	31.4 $\pm$ 0.2	39.6 $\pm$ 0.2
B	47.9 $\pm$ 0.2	27.9 $\pm$ 0.0	33.1 $\pm$ 0.7	38.3 $\pm$ 0.6	31.9 $\pm$ 0.9	40.0 $\pm$ 0.9
C	61.1 $\pm$ 0.1	47.4 $\pm$ 0.1	52.1 $\pm$ 0.5	54.1 $\pm$ 0.5	50.1 $\pm$ 0.4	55.2 $\pm$ 0.4
D	64.5 $\pm$ 0.1	58.3 $\pm$ 0.0	59.4 $\pm$ 0.8	59.6 $\pm$ 0.3	59.4 $\pm$ 0.9	60.9 $\pm$ 0.9
E	26.5 $\pm$ 0.1	21.7 $\pm$ 0.0	23.1 $\pm$ 0.8	22.5 $\pm$ 1.2	23.0 $\pm$ 0.7	24.5 $\pm$ 1.4

Appendix

Table 10: Full per-dataset results, pairs A1 and A3 (datasets by letter code, Table 3 ); mean

\pm

sd over three seeds.

	M-only	S-only	T2T	C2C	LoRA	MP@16
B1 : Gemma-4-31B $\to$ Gemma-4-12B ( $N_{M}/N_{S}=2.6$ )
A	65.6 $\pm$ 0.1	44.3 $\pm$ 0.0	50.4 $\pm$ 0.4	54.0 $\pm$ 0.4	48.2 $\pm$ 0.3	54.3 $\pm$ 0.3
B	51.5 $\pm$ 0.2	42.4 $\pm$ 0.0	45.4 $\pm$ 1.0	46.4 $\pm$ 0.4	44.5 $\pm$ 0.8	47.5 $\pm$ 0.8
C	68.0 $\pm$ 0.0	51.3 $\pm$ 0.2	56.3 $\pm$ 0.1	58.4 $\pm$ 0.1	54.2 $\pm$ 0.5	59.3 $\pm$ 0.7
D	54.2 $\pm$ 0.0	10.1 $\pm$ 0.2	11.9 $\pm$ 0.9	10.6 $\pm$ 0.6	13.1 $\pm$ 0.2	12.2 $\pm$ 1.2
E	31.5 $\pm$ 0.1	2.1 $\pm$ 0.0	3.9 $\pm$ 0.3	1.9 $\pm$ 0.3	3.8 $\pm$ 0.4	3.1 $\pm$ 0.7

Appendix

Table 11: Full per-dataset results, pairs B1 and C1 (see the B1 capability-limited note in Appendix F.2 ).

	M-only	S-only	T2T	C2C	LoRA	MP@16
C2 : Qwen3-32B $\to$ Qwen3-4B ( $N_{M}/N_{S}=8.2$ )
A	20.4 $\pm$ 0.0	13.0 $\pm$ 0.2	14.6 $\pm$ 0.4	16.1 $\pm$ 0.1	14.1 $\pm$ 0.3	15.9 $\pm$ 0.5
B	24.9 $\pm$ 0.1	17.8 $\pm$ 0.0	20.3 $\pm$ 1.7	20.4 $\pm$ 0.6	18.3 $\pm$ 0.6	21.7 $\pm$ 0.8
C	25.1 $\pm$ 0.1	14.7 $\pm$ 0.0	17.4 $\pm$ 0.3	19.2 $\pm$ 0.9	16.6 $\pm$ 0.1	19.7 $\pm$ 0.8
D	69.7 $\pm$ 0.2	60.1 $\pm$ 0.1	62.5 $\pm$ 0.8	63.0 $\pm$ 0.9	62.2 $\pm$ 0.7	64.3 $\pm$ 0.3
E	34.0 $\pm$ 0.0	24.8 $\pm$ 0.1	27.2 $\pm$ 1.1	26.4 $\pm$ 0.7	25.8 $\pm$ 1.2	29.0 $\pm$ 1.5

Appendix

Table 12: Full per-dataset results, pairs C2 and C3.

	M-only	S-only	T2T	C2C	LoRA	MP@16
D1 : Qwen3-14B $\to$ Qwen3-8B ( $N_{M}/N_{S}=1.8$ )
A	20.2 $\pm$ 0.1	6.1 $\pm$ 0.1	11.1 $\pm$ 0.3	13.9 $\pm$ 0.5	8.7 $\pm$ 0.4	13.5 $\pm$ 0.4
B	20.6 $\pm$ 0.1	13.5 $\pm$ 0.2	15.6 $\pm$ 1.0	16.6 $\pm$ 1.1	14.5 $\pm$ 1.8	18.4 $\pm$ 0.7
C	18.6 $\pm$ 0.1	12.0 $\pm$ 0.0	14.1 $\pm$ 0.9	15.3 $\pm$ 0.3	13.3 $\pm$ 0.8	15.4 $\pm$ 0.5
D	68.4 $\pm$ 0.0	63.5 $\pm$ 0.0	64.9 $\pm$ 0.6	66.0 $\pm$ 0.3	64.7 $\pm$ 0.8	66.6 $\pm$ 0.8
E	32.8 $\pm$ 0.2	28.6 $\pm$ 0.1	29.6 $\pm$ 1.0	29.4 $\pm$ 1.2	28.7 $\pm$ 0.6	30.8 $\pm$ 1.2

Appendix

Table 13: Full per-dataset results, pairs D1 and D2.

	M-only	S-only	T2T	C2C	LoRA	MP@16
D3 : Qwen3-14B $\to$ Qwen3-1.7B ( $N_{M}/N_{S}=8.7$ )
A	20.2 $\pm$ 0.1	9.5 $\pm$ 0.1	11.0 $\pm$ 0.5	12.7 $\pm$ 0.7	10.5 $\pm$ 0.3	12.3 $\pm$ 0.5
B	20.6 $\pm$ 0.1	9.7 $\pm$ 0.2	12.6 $\pm$ 1.1	12.4 $\pm$ 0.5	10.1 $\pm$ 1.1	12.9 $\pm$ 0.3
C	18.6 $\pm$ 0.1	10.2 $\pm$ 0.1	11.8 $\pm$ 0.9	12.6 $\pm$ 0.4	11.1 $\pm$ 0.3	12.3 $\pm$ 0.6
D	68.4 $\pm$ 0.0	56.6 $\pm$ 0.1	59.0 $\pm$ 0.5	58.6 $\pm$ 0.3	57.8 $\pm$ 0.2	60.1 $\pm$ 0.4
E	32.8 $\pm$ 0.2	20.7 $\pm$ 0.1	22.9 $\pm$ 0.7	23.2 $\pm$ 1.6	23.2 $\pm$ 1.5	24.5 $\pm$ 0.6

Appendix

Table 14: Full per-dataset results, pairs D3 and E1.

	$R{=}1$	$R{=}2$	$R{=}4$	$R{=}8$	$R{=}16$	$R{=}32$	$R{=}64$
A	36.5 $\pm$ 0.2	36.5 $\pm$ 0.3	36.7 $\pm$ 0.7	36.7 $\pm$ 0.8 ∗	36.7 $\pm$ 0.8 ∗	36.7 $\pm$ 0.8 ∗	36.7 $\pm$ 0.8 ∗
B	34.8 $\pm$ 0.5	35.2 $\pm$ 0.4	34.7 $\pm$ 1.5	34.9 $\pm$ 0.4	34.7 $\pm$ 0.4	33.9 $\pm$ 1.4	34.0 $\pm$ 1.5
C	51.0 $\pm$ 0.8	51.1 $\pm$ 0.7	50.9 $\pm$ 0.6	51.0 $\pm$ 0.4 ∗	51.0 $\pm$ 0.4 ∗	51.0 $\pm$ 0.4 ∗	51.0 $\pm$ 0.4 ∗
D	60.2 $\pm$ 0.6	60.5 $\pm$ 0.3	60.3 $\pm$ 0.5	60.0 $\pm$ 0.5	60.2 $\pm$ 0.3	59.7 $\pm$ 0.3	57.9 $\pm$ 0.8
E	24.2 $\pm$ 1.1	23.8 $\pm$ 0.4	23.8 $\pm$ 0.5	23.9 $\pm$ 1.1	23.7 $\pm$ 1.1	22.7 $\pm$ 1.2	21.5 $\pm$ 0.6
F	37.4 $\pm$ 0.9	37.1 $\pm$ 0.8	36.7 $\pm$ 0.4	37.2 $\pm$ 0.6	37.0 $\pm$ 1.2	37.1 $\pm$ 0.9	36.6 $\pm$ 0.4

Appendix

Table 15: Inference-time refresh-interval sweep on A2 (mean

\pm

sd). ∗ marks cells where the output ends before the first refresh boundary, so the condition is the static bridge exactly.

Pair	$R{=}1$	$R{=}2$	$R{=}4$	$R{=}8$	$R{=}16$	$R{=}32$	$R{=}64$
A1	56.8	56.4	56.6	55.4	54.0	44.6	29.7
A2 †	57.1	56.8	55.3	55.0	52.2	43.1	29.4
A3	37.9	37.5	37.5	36.8	35.7	29.2	23.8
B1	45.1	44.3	43.9	44.2	42.1	35.3	28.0
C1	57.5	56.9	57.6	56.6	54.6	43.5	31.3
C2	48.8	48.2	47.9	47.6	45.8	37.2	29.7

Appendix

Table 16: Macro-average recovery (%) against

R

for all eleven pairs. † The A2 row is derived from Table 15 and cross-checked against the registered sweep table by the derivation script.

Task	$L_{g}^{\prime}$	$R{=}8$	$R{=}16$	$R{=}32$	T2T ( $=R{\to}\infty$ )	MP@16
B GPQA	64	31.4 $\pm$ 0.9	31.6 $\pm$ 0.6	31.2 $\pm$ 0.7	28.3 $\pm$ 0.8	34.7 $\pm$ 0.4
	128	32.0 $\pm$ 0.5	31.9 $\pm$ 1.2	31.5 $\pm$ 0.5	28.3 $\pm$ 0.8	34.7 $\pm$ 0.4
D MATH-500	64	59.9 $\pm$ 0.9	59.7 $\pm$ 0.6	59.9 $\pm$ 0.2	59.4 $\pm$ 0.4	60.2 $\pm$ 0.3
	128	60.2 $\pm$ 0.6	59.9 $\pm$ 0.4	59.6 $\pm$ 0.2	59.4 $\pm$ 0.4	60.2 $\pm$ 0.3
E OlympiadBench	64	23.1 $\pm$ 1.0	22.7 $\pm$ 0.6	22.4 $\pm$ 0.3	21.9 $\pm$ 0.6	23.7 $\pm$ 1.1
	128	22.8 $\pm$ 1.1	23.0 $\pm$ 0.4	22.5 $\pm$ 1.0	21.9 $\pm$ 0.6	23.7 $\pm$ 1.1

Appendix

Table 17: rT2T full quality sweep on A2 (replace-style; mean

\pm

sd). Bold marks the per-task best

L_{g}^{\prime}

R{=}16

, the configuration entering Figure 4 . Short-answer sets (A, C, I, J) trigger no refresh and equal T2T cell by cell, as does the

R{\to}\infty

limit (double identity, hard-verified).

Pair	$V_{64}$ ( $\times 10^{-3}$ )	$H_{64}$ (nats)	MP macro rec. (%)
A1	3.03 $\pm$ 0.19	4.40 $\pm$ 0.07	54.0
A2	3.26 $\pm$ 0.05	4.26 $\pm$ 0.04	52.2
A3	2.38 $\pm$ 0.36	4.54 $\pm$ 0.06	35.7
B1	2.73 $\pm$ 0.20	4.44 $\pm$ 0.10	42.1
C1	3.10 $\pm$ 0.06	4.23 $\pm$ 0.03	54.6
C2	2.42 $\pm$ 0.08	4.58 $\pm$ 0.11	45.8

Appendix

Table 18: Read-distribution diagnostics for the eleven pairs: variance version

V_{64}

, entropy version

H_{64}

, and MP@16 macro recovery. Spearman

\rho(V_{64},\mathrm{rec.})=0.78

p=0.004

Code	Dataset (role)	Metric	Size	In. med.	Out. med.
D-IF	Multi-IF (primary: sign flip, PosQ)	constraint satisfaction	400	120	610
D-SUM	QMSum (long-in/long-out positive)	LLM-judge mean $\times$ 10	200	8200	640
D-WR	HelloBench (long writing) (open-ended positive)	LLM-judge mean $\times$ 10	150	190	1150
D-CODE	BigCodeBench (capability-limited negative)	pass@1	300	380	450
D-MCQ	ARC-Challenge (non-triggering negative)	accuracy	1170	460	9

Appendix

Table 19: The diagnostic suite (D suite): disjoint from the benchmarks A–M and from fuse_v3, selected by prior staleness sensitivity.

Condition	D-IF	D-SUM	D-WR	D-CODE	D-MCQ
M-only (ceiling)	73.2 $\pm$ 0.2	68.9 $\pm$ 0.9	71.3 $\pm$ 0.5	47.6 $\pm$ 0.1	92.8 $\pm$ 0.1
S-only (floor)	58.7 $\pm$ 0.0	59.4 $\pm$ 0.7	62.7 $\pm$ 0.6	30.8 $\pm$ 0.1	85.8 $\pm$ 0.1
OneShot (static bridge)	56.2 $\pm$ 0.3	58.9 $\pm$ 0.4	61.5 $\pm$ 0.5	30.5 $\pm$ 0.2	87.9 $\pm$ 0.6
OracleSwap @16	60.7 $\pm$ 0.5	61.6 $\pm$ 0.3	64.6 $\pm$ 0.2	31.1 $\pm$ 0.5	87.9 $\pm$ 0.6
OracleSwap-mis @16	55.5 $\pm$ 0.8	58.7 $\pm$ 0.2	61.0 $\pm$ 0.5	30.2 $\pm$ 0.9	87.9 $\pm$ 0.6
OneShot -mismatch	—	—	—	—	82.5 $\pm$ 0.4

Appendix

Table 20: Diagnostic condition matrix on A2 (mean

\pm

sd). OneShot -mismatch is collected only on D-MCQ (content canary); on triggering tasks attribution is carried by OracleSwap-mis @16.

Condition	First 1/3	Middle 1/3	Last 1/3	Aggregate
S-only	59.0 $\pm$ 0.3	58.6 $\pm$ 0.3	58.3 $\pm$ 0.4	58.7
OneShot	59.8 $\pm$ 0.3	56.4 $\pm$ 0.3	52.6 $\pm$ 0.8	56.2
OracleSwap @16	59.7 $\pm$ 0.8	60.5 $\pm$ 1.1	61.8 $\pm$ 0.6	60.7

Appendix

Table 21: Position-segment quality (PosQ) on D-IF; the aggregate column equals Table 20 .

$t$	0	64	128	256	384	512
stale $H_{\mathcal{M}}(x)$	0.79	0.73	0.69	0.58	0.53	0.47
fresh $H_{\mathcal{M}}(x\oplus\hat{y}_{<t})$	0.79	0.80	0.78	0.77	0.76	0.78
$\mathrm{StaleGap}(t)$	0.00	0.07	0.09	0.19	0.23	0.31

Appendix

Table 22: StaleGap probe readability on D-IF (linear readout, independent of the bridge).

\mathrm{StaleGap}(t)

is the fresh

-

stale difference.

Layers $k$	$\mathrm{Bytes}(16)$	Transfer	$\tau_{\mathrm{sync}}$ (w/ RTT)
8	1.33 MB	71 ms	89 ms
16 (main)	2.66 MB	142 ms	160 ms
32	5.32 MB	284 ms	302 ms
Initial memory ( $p_{x}{=}128$ , $k{=}16$ )		13.1 MB
Full fused KV (C2C, 2K input)		${\sim}268$ MB

Appendix

Table 23: Transfer accounting at

R{=}16

for A2 (

d_{t}{=}2560

; 150 Mbps cross-site link, 18 ms RTT). The

k{=}16

row is the deployed configuration (ablation row 11).

Model	Hardware	$P$	$D$	$D_{b}$
Qwen3.5-27B (mentor)	H100 80GB	8,600	54	—
Qwen3.5-4B (student)	RTX 4090 24GB	23,500	121	104

Appendix

Table 24: Measured serving throughput (vLLM 0.19.1, bf16, batch size 1, 8K context).

D_{b}

is decoding with the bridge attached.

Hardware	$/h	Source	Quoted
H100 80GB (mentor)	2.79	Lambda on-demand	2026-07-10
RTX 4090 (student)	0.44	RunPod community	2026-07-10

Appendix

Table 25: GPU rental prices used in the cost model.

Task	T2T	MP@1	MP@2	MP@4	MP@8	MP@16	MP@32	MP@64	MP@ $\infty$
A	1.91	0.15	0.09	0.07	0.05	0.05	0.05	0.05	0.05
J	3.44	0.59	0.53	0.51	0.49	0.49	0.49	0.49	0.49
H	4.55	20.14	10.66	5.92	3.56	2.37	1.79	1.49	1.20
K	5.89	15.81	8.72	5.17	3.39	2.50	2.06	1.84	1.62

Appendix

Table 26: Per-request cost (

10^{-3}

dollars, blocking accounting) for the four representative tasks of Figure 8 .

Task	$L_{g}^{\prime}$	$R{=}4$	$R{=}8$	$R{=}16$	$R{=}32$	$R{=}64$	$\infty$	T2T	MP@16
H WritingBench	64	254.8	129.7	66.7	35.6	19.6	$=$ T2T	4.55	2.37
	128	503.7	254.2	128.3	66.5	34.5	$=$ T2T	4.55	2.37
K GovReport	64	200.8	103.4	54.7	30.3	18.1	$=$ T2T	5.89	2.50
	128	388.0	197.0	101.5	53.7	29.8	$=$ T2T	5.89	2.50

Appendix

Table 27: rT2T per-request cost (

10^{-3}

dollars, replace-style, blocking; analytically computed from Eq. 6 and the measured inputs). Bold marks each task’s quality-best configuration (Table 17 ). At

\infty

n_{\mathrm{sync}}{=}0

and rT2T equals T2T identically, as on the short-answer tasks (which therefore carry no rows).

Figure 9: Per-dataset recovery for all eleven pairs (rows, grouped by mentor group) on all thirteen datasets (columns) under the four methods; color is recovery clipped to $[-20,100]$ as in Figure 5 , red marking harm. $\times$ marks the two capability-limited B1 cells registered in Appendix F.2 . Numbers behind every cell are in Tables 8 and 10 – 14 .

Figure 10: StaleGap probe on D-IF (enlarged version of Figure 2 b; linear readout on frozen mentor states, independent of the bridge). Top: fresh and stale probe readability over generation position; the shaded wedge is

\mathrm{StaleGap}(t)

and the bracket marks the 0.32 decay depth. Bottom:

\mathrm{StaleGap}(t)

values (fresh

-

stale, Table 22 ).

Figure 11: Stage-2 outlook (illustrative pilot; no quantitative claim). Background: density of offline refresh-value labels (CE difference between decoding under stale and fresh memory). Bars: where the learned trigger fires at the

R{=}16

equal budget; the dashed line is the uniform schedule. Firing concentrates at constraint switches; the inset shows the event-aligned rate on the

R{=}16

window scale.

Figure 12: Two illustrative cases (more in the full logs, to be released). Top: on long instruction following, static guidance points the student back to a satisfied constraint (gray underline and arrow) and a later constraint is missed (red); with

R{=}16

the refreshed memory reflects the text written so far and the remaining constraint is completed (purple). Bottom: on a capability-limited coding task both conditions produce the same flawed algorithm — fresh guidance does not create missing capability.

Memory-Augmented Language Models

Memory-augmented language models combine persistent or temporary memory with neural computation to retain, retrieve, and use information beyond the active context.

See all 291 papers

LongCrafter: Towards Diverse Long-Context Understanding via Evidence-Graph-Guided Instruction Synthesis
Jul 7, 2026Chenhao Yuan, Yinhao Xu, Shuwen Xu +8Lost-in-the-Middle Effect LLM Fine-Tuning
University of Chinese Academy of Sciences, Beijing, China · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
IS-CoT: Breaking the Long-form Generation Collapse via Interleaved Structural Thinking
Jun 8, 2026Zechen Sun, Yuyang Sun, Zecheng Tang +6Open-Ended Generation Long-Form Document Generation
Institute of Computer Science and Technology, Soochow University · 2Information Research Center of Military Science, PLA Academy of Military Science
Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026
Jun 3, 2026Enes Yavuz Ugan, Maike Züfle, Yuka Ko +5MBR Decoding Instruction Following
1Karlsruhe Institute of Technology · 2Carnegie Mellon University

Jul 7, 2026J/K move · Enter open · S save

LongCrafter: Towards Diverse Long-Context Understanding via Evidence-Graph-Guided Instruction Synthesis

Chenhao Yuan, Yinhao Xu, Shuwen Xu, Xizhi Yang+7

University of Chinese Academy of Sciences, Beijing, China · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China

Lost-in-the-Middle Effect LLM Fine-Tuning Evidence-Grounded Generation Synthetic Data Generation for Language Models

Open paper AskPDF arXiv

Synthesizing long-context supervised fine-tuning (SFT) data is a scalable way to enhance the long-context understanding of large language models (LLMs), yet existing approaches share three limitations: narrow task coverage, insufficient instruction difficulty, and a lack of faithfulness supervision. We propose LongCrafter, a structured synthesis framework that couples a hierarchical task taxonomy with an evidence-grounded pipeline. The taxonomy organizes long-context understanding into local/shallow and global/deep levels and yields 32 fine-grained task types that serve as a global generative prior. Guided by this taxonomy, LongCrafter constructs task-aligned long contexts, decomposes them into explicit evidence graphs that model cross-paragraph dependencies, and generates instruction--response pairs strictly grounded in the located evidence spans, ensuring both controllable difficulty and faithful, traceable reasoning. Models fine-tuned on LongCrafter data outperform all SFT baselines and even the official post-trained models on LongBench, LongBench~v2, and LooGLE across both Qwen2.5-7B and LLaMA-3.1-8B, with the largest gains on high-difficulty tasks. Further analysis shows that LongCrafter data is more diverse and better spread across difficulty levels, and that the trained models locate evidence robustly regardless of position, effectively mitigating the ``lost in the middle'' problem.

Abstract

Memory-Augmented Language Models

Explore similar work

LongCrafter: Towards Diverse Long-Context Understanding via Evidence-Graph-Guided Instruction Synthesis

IS-CoT: Breaking the Long-form Generation Collapse via Interleaved Structural Thinking

Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026