Encoding Context into LLM Weights: What Makes It Context?

Yue Wang; Qizhou Wang; Feng Liu; Gang Niu; Masashi Sugiyama; Bo Han

cs.CLOct 13, 2026

Encoding Context into LLM Weights: What Makes It Context?

Authors: Yue Wang, Qizhou Wang, Feng Liu, Gang Niu, Masashi Sugiyama, Bo Han

Organizations: The University of Melbourne · RIKEN Center for Advanced Intelligence Project · The University of Tokyo · TMLR Group, Hong Kong Baptist University

Abstract

Longer contexts allow large language models (LLMs) to condition their predictions on more information, but inference costs grow prohibitively with context length. An emerging alternative uses test-time updates to encode context as temporary changes to model weights. However, encoding this information into weights that already represent general knowledge obscures its role as context, so the LLM may fail to use it properly to condition subsequent inference. To restore this contextual role, we first associate each context with an identifier during encoding, thereby distinguishing the newly encoded information from pre-existing knowledge. Invoking the identifier at inference then enables the LLM to recite the encoded context. Nevertheless, successful recitation does not ensure that the LLM will use the recalled information as context. We therefore further introduce a prompting framework that explicitly guides the LLM to use this information as context. Across diverse controlled tasks and practical benchmarks, our method outperformed the baseline using only off-the-shelf LLMs, showing its advances in context encoding.

Figures & tables

Figure 1: Comparison of (a) full-context inference, (b) encoded-context inference with meta-trained TCE, and (c) encoded-context inference with ACE . Full-context inference jointly processes $c_{1:T}$ and $x$ , becoming infeasible when their combined length exceeds the input context window. TCE and ACE instead encode the chunks $c_{1:T}$ into LLM weights, illustrated here as FFN weights. Prior TCE methods often rely on additional meta-training-based fine-tuning, whereas ACE operates directly on off-the-shelf LLMs without such fine-tuning.

LLM	Method	Prompt	Recall	Recognition	Continuation	Retention
Qwen3.5-4B	TCE	–	6.1	0.5	0.0	100.0
	ACE	ci_short	52.3	91.9	46.4	99.0
	ACE	ci_v1	55.3	81.7	48.8	100.0
	ACE	ci_v2	60.4	80.7	28.0	100.0
Qwen3.5-9B	TCE	–	4.6	2.0	1.2	100.0
	ACE	ci_short	40.1	96.4	9.5	100.0

Table 1: Comparison between TCE and ACE on Recall, Recognition, Continuation, and Retention. We reported accuracy (%) on the Qwen3.5 family. Bold marks the best result within each model and task.

LLM	Method	Prompt	Counterfact	Dispute	Outdated	Counterfact
LLM	Method	Prompt	Counterfact	Dispute	Outdated	(two-sided)
Qwen3.5-4B	TCE	–	3.5	7.0	5.9	0.5
	ACE	ci_short	86.5	83.7	88.2	20.0
	ACE	ci_v1	92.5	95.3	100.0	27.5
	ACE	ci_v2	93.0	90.7	94.1	28.0
Qwen3.5-9B	TCE	–	14.0	11.6	10.0	6.0

Table 2: Comparison between TCE and ACE on the three categories of Conflict: Counterfactuals (Counterfact), Disputes (Dispute), and Outdated Information (Outdated). We further reported two-sided Counterfactual accuracy in the column labeled “Counterfact (two-sided).”

LLM	Method	RULER			LongBench	LongBench v2
LLM	Method	NIAH-1	NIAH-2	NIAH-3	LongBench	LongBench v2
Qwen3.5-4B	Gold Standard	100	100	100	20.8	2.5
	TCE	98	7	0	8.8	0.0
	ACE	100	35	15	9.0	2.0
Qwen3.5-9B	Gold Standard	100	100	100	20.3	3.0
	TCE	88	7	1	8.2	2.5

Table 3: Comparison between TCE and ACE on RULER , LongBench , and LongBench v2 . We reported retrieval accuracy (%) on RULER , answer F1 (

\times 100

) on LongBench , and summed judge-assigned scores over

100

instances on LongBench v2 . By default, we use ci_v2. Gold Standard refers to full-context inference with the context retained in the input. Bold marks the better result between TCE and ACE .

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

LLM	Method	Cue	Recall	Recognition
Qwen3.5-4B	TCE	–	6.1	0.5
	TCE-AM	–	7.1	10.7
	TCE-AM	UID	5.6	35.5
	ACE	ci_short	52.3	91.9
Qwen3.5-9B	TCE	–	4.6	2.0
	TCE-AM	–	1.5	3.6

Appendix

Table 4: Comparison between TCE, TCE-AM , and ACE on Recall and Recognition. We reported accuracy (%) on the Qwen3.5 family. Bold marks the best result among TCE and TCE-AM within each model and task.

LLM	Method	Recall	Recognition
Qwen3.5-4B	ACE w/o $\mathcal{L}_{\mathrm{A}}$	4.1	21.3
	ACE w/o $\mathcal{L}_{\mathrm{B}}$	37.1	49.7
	ACE with TCE inference	32.5	81.2
	ACE w/o encoding	0.0	0.0
	ACE	55.3	81.7
Qwen3.5-9B	ACE w/o $\mathcal{L}_{\mathrm{A}}$	7.6	17.3

Appendix

Table 5: Ablation of ACE on Recall and Recognition. We reported accuracy (%) on the Qwen3.5 family. Bold marks the best result within each model and task.

Method	Fast Weights	Recall	Recognition	Continuation	Retention	Counterfact	Dispute
TCE	FFN-1	6.1	0.5	0.0	100.0	3.5	7.0
	FFN-2	4.1	7.6	0.0	68.0	3.0	7.0
	FFN-3	9.1	4.1	3.6	43.0	9.5	4.7
	ATTN-1	5.6	0.0	0.0	84.0	0.5	2.3
	ATTN-2	11.7	0.0	0.0	74.0	1.0	2.3
	ATTN-3	9.6	0.0	0.0	69.0	2.5	2.3

Appendix

Table 6: Comparison between TCE and ACE with different choices of fast weights on Qwen3.5-4B. FFN-

k

and ATTN-

k

denote the FFN and attention sublayers of the final

k

transformer layers, respectively. We reported accuracy (%) with ACE using ci_v1. Bold marks the best result within each method and task.

Method	Fast Weights	Recall	Recognition	Continuation	Retention	Counterfact	Dispute
TCE	FFN-1	4.6	2.0	1.2	100.0	14.0	11.6
	FFN-2	10.2	0.5	26.2	100.0	26.5	11.6
	FFN-3	17.3	0.5	26.8	100.0	10.5	7.0
	ATTN-1	14.2	0.0	0.6	100.0	19.5	14.0
	ATTN-2	17.8	0.0	2.4	100.0	31.0	14.0
	ATTN-3	23.9	0.0	2.4	100.0	35.5	16.3

Appendix

Table 7: Comparison between TCE and ACE with different choices of fast weights on Qwen3.5-9B. FFN-

k

and ATTN-

k

denote the FFN and attention sublayers of the final

k

transformer layers, respectively. We reported accuracy (%) with ACE using ci_v1. Bold marks the best result within each method and task.

$\tau$	Method	Prompt	Recall	Recognition	Continuation	Retention
0.1	TCE	–	6.1	0.5	0.0	100.0
	ACE	ci_short	52.3	91.9	46.4	99.0
	ACE	ci_v1	55.3	81.7	48.8	100.0
	ACE	ci_v2	60.4	80.7	28.0	100.0
0.2	TCE	–	6.6	0.0	0.0	100.0
	ACE	ci_short	51.3	86.3	45.8	99.0

Appendix

Table 8: Comparison between TCE and ACE on Recall, Recognition, Continuation, and Retention across stopping thresholds. We reported accuracy (%) on Qwen3.5-4B. Bold marks the best result within each threshold and task.

$\tau$	Method	Prompt	Counterfact	Dispute	Outdated	Counterfact
$\tau$	Method	Prompt	Counterfact	Dispute	Outdated	(two-sided)
0.1	TCE	–	3.5	7.0	5.9	0.5
	ACE	ci_short	86.5	83.7	88.2	20.0
	ACE	ci_v1	92.5	95.3	100.0	27.5
	ACE	ci_v2	93.0	90.7	94.1	28.0
0.2	TCE	–	4.5	4.7	5.9	1.5

Appendix

Table 9: Comparison between TCE and ACE on the three categories of Conflict across stopping thresholds. We reported accuracy (%) on Qwen3.5-4B, together with two-sided Counterfactual accuracy. Bold marks the best result within each threshold and metric.

$\tau$	Method	Prompt	Recall	Recognition	Continuation	Retention
0.1	TCE	–	4.6	2.0	1.2	100.0
	ACE	ci_short	40.1	96.4	9.5	100.0
	ACE	ci_v1	35.5	95.4	13.7	100.0
	ACE	ci_v2	38.1	94.4	5.4	100.0
0.2	TCE	–	4.1	0.5	0.6	100.0
	ACE	ci_short	37.1	86.8	8.3	100.0

Appendix

Table 10: Comparison between TCE and ACE on Recall, Recognition, Continuation, and Retention across stopping thresholds. We reported accuracy (%) on Qwen3.5-9B. Bold marks the best result within each threshold and task.

$\tau$	Method	Prompt	Counterfact	Dispute	Outdated	Counterfact
$\tau$	Method	Prompt	Counterfact	Dispute	Outdated	(two-sided)
0.1	TCE	–	14.0	11.6	10.0	6.0
	ACE	ci_short	94.0	95.3	100.0	29.0
	ACE	ci_v1	94.5	95.3	100.0	30.5
	ACE	ci_v2	95.0	97.7	100.0	30.5
0.2	TCE	–	15.0	11.6	10.0	10.0

Appendix

Table 11: Comparison between TCE and ACE on the three categories of Conflict across stopping thresholds. We reported accuracy (%) on Qwen3.5-9B, together with two-sided Counterfactual accuracy. Bold marks the best result within each threshold and metric.

$\tau$	Method	Prompt	Recall	Recognition	Continuation	Retention
0.1	TCE	–	1.5	0.0	0.0	100.0
	ACE	ci_short	75.6	98.0	20.8	100.0
	ACE	ci_v1	39.1	58.4	11.9	100.0
	ACE	ci_v2	45.2	50.8	2.4	100.0
0.2	TCE	–	2.0	0.0	0.0	100.0
	ACE	ci_short	71.6	83.8	0.6	100.0

Appendix

Table 12: Comparison between TCE and ACE on Recall, Recognition, Continuation, and Retention across stopping thresholds. We reported accuracy (%) on Qwen3.5-27B. Bold marks the best result within each threshold and task.

$\tau$	Method	Prompt	Counterfact	Dispute	Outdated	Counterfact
$\tau$	Method	Prompt	Counterfact	Dispute	Outdated	(two-sided)
0.1	TCE	–	11.0	18.6	54.5	8.0
	ACE	ci_short	96.0	97.7	97.0	36.5
	ACE	ci_v1	93.0	88.4	100.0	29.0
	ACE	ci_v2	92.0	93.0	97.0	31.5
0.2	TCE	–	41.0	18.6	54.5	36.0

Appendix

Table 13: Comparison between TCE and ACE on the three categories of Conflict across stopping thresholds. We reported accuracy (%) on Qwen3.5-27B, together with two-sided Counterfactual accuracy. Bold marks the best result within each threshold and metric.

LLM	Method	Prompt	$\tau=0.1$	$\tau=0.2$	$\tau=1.0$
Qwen3.5-4B	TCE	–	0.0	0.0	0.0
	ACE	ci_short	50.0	48.2	38.1
	ACE	ci_v1	53.6	52.4	49.4
	ACE	ci_v2	31.5	31.0	28.6
Qwen3.5-9B	TCE	–	1.2	0.6	0.0
	ACE	ci_short	28.0	21.4	1.2

Appendix

Table 14: Comparison between TCE and ACE on question recovery in Continuation across stopping thresholds. We reported question-recovery rates (%) on the Qwen3.5 family. These scores provide a lower bound on retrieval, rather than measuring whether the encoded question was answered correctly. Bold marks the best result within each model and threshold.

LLM	Method	Prompt	Dispute control			Counterfact control
LLM	Method	Prompt	$\tau=0.1$	$\tau=0.2$	$\tau=1.0$	$\tau=0.1$	$\tau=0.2$	$\tau=1.0$
Qwen3.5-4B	TCE	–	74.4	79.1	81.4	64.0	68.0	81.0
	ACE	ci_short	79.1	81.4	79.1	76.5	77.0	80.0
	ACE	ci_v1	81.4	81.4	79.1	76.0	77.0	81.5
	ACE	ci_v2	81.4	81.4	81.4	79.0	80.0	82.0
Qwen3.5-9B	TCE	–	81.4	81.4	81.4	82.5	82.5	82.5

Appendix

Table 15: Comparison between TCE and ACE on non-conflicting control queries paired with Dispute and Counterfactual instances. We reported accuracy (%) on the Qwen3.5 family across stopping thresholds. Bold marks the best result within each model, threshold, and control category.

Updates $K$	Method	Recall	Recognition	NIAH-2
2	TCE	3.0	0.0	0.0
2	ACE	15.2	0.0	0.0
4	TCE	1.5	1.5	0.0
4	ACE	45.7	62.4	1.0
8	TCE	2.0	27.9	8.0
8	ACE	58.9	98.5	16.0

Appendix

Table 16: Comparison between TCE and ACE across fixed update budgets on Qwen3.5-4B. We reported accuracy (%) on Recall, Recognition, and RULER NIAH-2, with ACE using ci_v1. Bold marks the best result within each budget and task.

Updates $K$	Method	Recall	Recognition	NIAH-2
2	TCE	3.0	0.0	0.0
2	ACE	10.2	0.0	0.0
4	TCE	4.1	1.0	0.0
4	ACE	34.0	79.2	1.0
8	TCE	0.5	70.6	5.0
8	ACE	44.7	99.5	8.0

Appendix

Table 17: Comparison between TCE and ACE across fixed update budgets on Qwen3.5-9B. We reported accuracy (%) on Recall, Recognition, and RULER NIAH-2, with ACE using ci_v1. Bold marks the best result within each budget and task.

Updates $K$	Method	Recall	Recognition	NIAH-2
2	TCE	0.5	0.0	0.0
2	ACE	12.7	0.0	0.0
4	TCE	2.0	0.0	0.0
4	ACE	40.1	15.2	0.0
8	TCE	2.0	6.6	5.0
8	ACE	42.6	99.0	21.0

Appendix

Table 18: Comparison between TCE and ACE across fixed update budgets on Qwen3.5-27B. We reported accuracy (%) on Recall, Recognition, and RULER NIAH-2, with ACE using ci_v1. Bold marks the best result within each budget and task.

LLM	Contextual answer		Prior answer ( ACE )
LLM	TCE	ACE	With invocation	Without invocation
Qwen3.5-4B	3.5	86.5	25.5	31.5
Qwen3.5-9B	14.0	94.0	33.5	29.0
Qwen3.5-27B	11.0	96.0	39.0	35.5

Appendix

Table 19: Comparison between TCE and ACE on contextual answering and the effect of invocation on prior-knowledge recovery. We reported accuracy (%) on

200

Counterfactual instances per model with

\tau=0.1

. TCE used no invocation, while ACE used ci_short. The two prior-answer conditions used the same encoded model state and query.

LLM	Method	Recall	Recognition	NIAH-2
Qwen3.5-4B	TCE	6.1	0.5	7.0
	TCE with generic invocation	2.0	0.0	10.0
	ACE	55.3	81.7	35.0
Qwen3.5-9B	TCE	4.6	2.0	7.0
	TCE with generic invocation	9.1	1.5	7.0
	ACE	35.5	95.4	17.0

Appendix

Table 20: Comparison between TCE, TCE with generic invocation, and ACE on Recall, Recognition, and RULER NIAH-2. We reported accuracy (%) on the Qwen3.5 family with

\tau=0.1

. Bold marks the best result within each model and task.

LLM	ACE with shared UID	ACE
Qwen3.5-4B	66.0	81.7
Qwen3.5-9B	91.9	95.4
Qwen3.5-27B	47.7	58.4

Appendix

Table 21: Comparison between ACE with a shared UID and the original ACE on Recognition. We reported accuracy (%) on the Qwen3.5 family with

\tau=0.1

. The shared UID was the fixed string memory. Bold marks the better result within each model.

LLM	Method	Recall	Recognition	NIAH-2
Qwen3.5-4B	ACE with alternative wording	54.3	28.9	33.0
Qwen3.5-4B	ACE	55.3	81.7	32.0
Qwen3.5-9B	ACE with alternative wording	34.5	77.7	14.0
Qwen3.5-9B	ACE	35.5	95.4	15.0
Qwen3.5-27B	ACE with alternative wording	41.6	76.1	47.0
Qwen3.5-27B	ACE	39.1	58.4	39.0

Appendix

Table 22: Comparison between ACE with alternative encoding wording and the original ACE on Recall, Recognition, and RULER NIAH-2. We reported accuracy (%) on the Qwen3.5 family with

\tau=0.1

. Bold marks the best result within each model and task.

LLM	$\tau$	Method	Prompt	NIAH-1	NIAH-2	NIAH-3
Qwen3.5-4B	0.1	TCE	–	98.0	7.0	0.0
		ACE	ci_short	99.0	30.0	18.0
		ACE	ci_v1	100.0	32.0	13.0
		ACE	ci_v2	100.0	35.0	15.0
	0.2	TCE	–	100.0	8.0	0.0
		ACE	ci_short	99.0	26.0	19.0

Appendix

Table 23: Comparison between TCE and ACE on RULER across stopping thresholds for Qwen3.5-4B and Qwen3.5-9B. We reported retrieval accuracy (%). The three ACE prompts were applied to the same encoded state within each model, task, and threshold. Bold marks the best result within each model, threshold, and task.

LLM	$\tau$	Method	Prompt	NIAH-1	NIAH-2	NIAH-3
Qwen3.5-27B	0.1	TCE	–	85.0	7.0	0.0
		ACE	ci_short	100.0	45.0	33.0
		ACE	ci_v1	100.0	39.0	31.0
		ACE	ci_v2	100.0	40.0	29.0
	0.2	TCE	–	80.0	9.0	0.0
		ACE	ci_short	99.0	41.0	33.0

Appendix

Table 24: Comparison between TCE and ACE on RULER across stopping thresholds for Qwen3.5-27B and Muse-Glimmer-30B. We reported retrieval accuracy (%). The three ACE prompts were applied to the same encoded state within each model, task, and threshold. Bold marks the best result within each model, threshold, and task.

LLM	Method	$\tau=5$	$\tau=8$	$\tau=10$
Qwen3.5-4B	Gold Standard	20.8	20.8	20.8
	TCE	7.0	8.8	8.6
	ACE	7.8	9.0	10.0
Qwen3.5-9B	Gold Standard	20.3	20.3	20.3
	TCE	7.4	8.2	9.2
	ACE	10.8	9.6	10.1

Appendix

Table 25: Comparison between TCE and ACE on LongBench across stopping thresholds. We reported answer F1 (

\times 100

) under sequential per-chunk encoding. Gold Standard refers to full-context inference, which does not depend on

\tau

. Bold marks the better result between TCE and ACE within each model and threshold.

$\tau$	Method	Prompt	NIAH-1	NIAH-2	NIAH-3
0.1	TCE	–	71.0	76.0	75.3
	ACE	ci_short	72.3	75.8	75.3
	ACE	ci_v1	72.0	73.8	73.3
	ACE	ci_v2	71.0	72.5	72.8
0.2	TCE	–	71.5	76.0	75.5
	ACE	ci_short	72.8	75.5	75.0

Appendix

Table 26: Comparison between TCE and ACE on general capability retention with Qwen3.5-4B. We reported MMLU-200 accuracy (%) after encoding contexts from each RULER task. Bold marks the best result within each threshold and task.

$\tau$	Method	Prompt	NIAH-1	NIAH-2	NIAH-3
0.1	TCE	–	75.5	77.0	76.5
	ACE	ci_short	67.0	74.5	73.5
	ACE	ci_v1	62.5	72.5	70.0
	ACE	ci_v2	58.8	71.0	67.5
0.2	TCE	–	75.8	77.3	76.8
	ACE	ci_short	68.8	74.5	74.0

Appendix

Table 27: Comparison between TCE and ACE on general capability retention with Qwen3.5-9B. We reported MMLU-200 accuracy (%) after encoding contexts from each RULER task. Bold marks the best result within each threshold and task.

$\tau$	Method	Prompt	NIAH-1	NIAH-2	NIAH-3
0.1	TCE	–	79.3	80.0	80.3
	ACE	ci_short	82.0	81.8	81.5
	ACE	ci_v1	80.8	80.8	80.8
	ACE	ci_v2	81.0	80.5	79.5
0.2	TCE	–	79.3	80.0	80.3
	ACE	ci_short	82.0	81.8	81.3

Appendix

Table 28: Comparison between TCE and ACE on general capability retention with Qwen3.5-27B. We reported MMLU-200 accuracy (%) after encoding contexts from each RULER task. Bold marks the best result within each threshold and task.

$\tau$	Method	Prompt	NIAH-1	NIAH-2	NIAH-3
0.1	TCE	–	66.3	72.8	69.5
	ACE	ci_short	76.5	76.0	76.0
	ACE	ci_v1	75.5	75.0	75.3
	ACE	ci_v2	76.8	76.8	76.5
0.2	TCE	–	66.3	71.3	64.3
	ACE	ci_short	76.5	76.0	76.3

Appendix

Table 29: Comparison between TCE and ACE on general capability retention with Muse-Glimmer-30B. We reported MMLU-200 accuracy (%) after encoding contexts from each RULER task. Bold marks the best result within each threshold and task.

LLM	Correct UID	Decoy UID
Qwen3.5-4B	87.0	1.0
Qwen3.5-9B	84.0	8.0
Qwen3.5-27B	100.0	4.0

Appendix

Table 30: UID ablation for ACE on NIAH-1 in RULER . We reported retrieval accuracy (%) on the Qwen3.5 family. Both conditions use the same encoded state, with only the correct or decoy UID as the addressing cue at inference. Bold marks the better result within each model.

LLM	Gold Standard	TCE	ACE
Qwen3.5-9B	100.0	30.0	80.0
Qwen3.5-27B	100.0	40.0	70.0
Muse-Glimmer-30B	100.0	20.0	30.0

Appendix

Table 31: Comparison between TCE and ACE on RULER NIAH-1 with

64

K contexts. We reported retrieval accuracy (%). Gold Standard denotes full-context inference. Bold marks the better encoding method within each model.

LLM	NIAH-2			NIAH-3
LLM	Gold Standard	TCE	ACE	Gold Standard	TCE	ACE
Qwen3.5-4B	100.0	3.3	6.7	100.0	0.0	3.3
Qwen3.5-9B	100.0	6.7	16.7	100.0	0.0	3.3
Qwen3.5-27B	100.0	3.3	20.0	100.0	0.0	13.3
Muse-Glimmer-30B	100.0	10.0	16.7	100.0	3.3	20.0

Appendix

Table 32: Comparison between TCE and ACE on RULER NIAH-2 and NIAH-3 with

4

K contexts. We reported retrieval accuracy (%). Gold Standard denotes full-context inference. Bold marks the better encoding method within each model and task.

Dataset	Method	Encoding	Steps	Throughput	Response	Output
Dataset	Method	time (s)	per chunk	(tokens/s)	latency (s)	tokens
Sandbox	Full context	–	–	–	0.64	11
	TCE	1.10	6.0	15	4.74	123
	ACE	1.34	7.0	13	0.88	17
RULER NIAH-1	Full context	–	–	–	0.63	9
	TCE	10.08	40.0	82	4.89	128

Appendix

Table 33: Comparison of computational cost between full-context inference, TCE, and ACE on Qwen3.5-4B. We reported encoding time, steps per chunk, encoding throughput, response latency, and output length on one fixed example per benchmark family, with averaging as described in the text. Output lengths are rounded to integers.

Dataset	Method	Encoding	Steps	Throughput	Response	Output
Dataset	Method	time (s)	per chunk	(tokens/s)	latency (s)	tokens
Sandbox	Full context	–	–	–	0.68	11
	TCE	0.98	5.0	17	3.63	90
	ACE	1.18	6.0	14	2.85	72
RULER NIAH-1	Full context	–	–	–	0.66	9
	TCE	7.31	28.0	113	4.98	128

Appendix

Table 34: Comparison of computational cost between full-context inference, TCE, and ACE on Qwen3.5-9B. We reported encoding time, steps per chunk, encoding throughput, response latency, and output length on one fixed example per benchmark family, with averaging as described in the text. Output lengths are rounded to integers.

Dataset	Method	Encoding	Steps	Throughput	Response	Output
Dataset	Method	time (s)	per chunk	(tokens/s)	latency (s)	tokens
Sandbox	Full context	–	–	–	1.11	11
	TCE	1.92	5.0	9	6.82	87
	ACE	2.44	6.0	7	5.95	73
RULER NIAH-1	Full context	–	–	–	1.25	9
	TCE	11.97	21.0	69	9.72	128

Appendix

Table 35: Comparison of computational cost between full-context inference, TCE, and ACE on Qwen3.5-27B. We reported encoding time, steps per chunk, encoding throughput, response latency, and output length on one fixed example per benchmark family, with averaging as described in the text. Output lengths are rounded to integers.

Dataset	Method	Encoding	Steps	Throughput	Response	Output
Dataset	Method	time (s)	per chunk	(tokens/s)	latency (s)	tokens
RULER NIAH-1	Full context	–	–	–	0.57	5
	TCE	3.36	10.0	235	0.14	1
	ACE	9.56	15.0	82	7.15	128
LongBench	Full context	–	–	–	5.55	64
	TCE	5.87	1.7	1009	3.87	64

Appendix

Table 36: Comparison of computational cost between full-context inference, TCE, and ACE on Muse-Glimmer-30B. We reported encoding time, steps per chunk, encoding throughput, response latency, and output length on one fixed example per benchmark family, with averaging as described in the text. Output lengths are rounded to integers.

LLM	Dataset	Encoding		Inference
LLM	Dataset	TCE	ACE	Full context	TCE	ACE
Qwen3.5-4B	Sandbox	1.5	13.7	0.4	1.22	0.83
	RULER NIAH-1	336.5	926.4	6.6	1.43	1.94
	LongBench	326.1	929.6	44.9	0.75	2.26
	LongBench v2	1003.6	2522.3	131.9	2.07	6.05
Qwen3.5-9B	Sandbox	2.4	22.1	0.8	1.79	2.47

Appendix

Table 37: Comparison of computational cost between full-context inference, TCE, and ACE . We reported estimated encoding and inference FLOPs (TFLOPs) on one fixed example per benchmark family. Inference costs were averaged over the queries associated with each example.

LLM	Dataset	Method	KV cache (MiB)
Qwen3.5-4B	Sandbox	Full context	1.4
		TCE	0.8
		ACE	3.0
	RULER NIAH-1	Full context	28.2
		TCE	1.6
		ACE	3.8

Appendix

Table 38: Comparison of inference KV-cache storage between full-context inference, TCE, and ACE on Qwen3.5-4B and Qwen3.5-9B. We reported average KV-cache storage (MiB) immediately after prompt prefill on one fixed example per benchmark family.

LLM	Dataset	Method	KV cache (MiB)
Qwen3.5-27B	Sandbox	Full context	2.8
		TCE	1.7
		ACE	6.1
	RULER NIAH-1	Full context	56.5
		TCE	3.1
		ACE	7.6

Appendix

Table 39: Comparison of inference KV-cache storage between full-context inference, TCE, and ACE on Qwen3.5-27B and Muse-Glimmer-30B. We reported average KV-cache storage (MiB) immediately after prompt prefill on one fixed example per benchmark family.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 246 papers

ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
Jul 2, 2026Yanjun Zhao, Ruizhong Qiu, Tianxin Wei +6Long-Context Inference Long-Context Reasoning in Language Models
University of Illinois Urbana-Champaign
RecaLLM: Addressing the Lost-in-Thought Phenomenon with Explicit In-Context Retrieval
Apr 10, 2026Kyle Whitecross, Negin RahimiRetrieval-Augmented Reasoning Long-Context Reasoning in Language Models
University of Massachusetts Amherst
Emergence of Context Characteristics Sensitivity in Large Language Models
Jun 8, 2026Nadya Yuki Wangsajaya, Haeun Yu, Isabelle AugensteinLLM Fine-Tuning Instruction Tuning
Nanyang Technological University · University of Copenhagen

Jul 2, 2026J/K move · Enter open · S save

ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning

Yanjun Zhao, Ruizhong Qiu, Tianxin Wei, Yuanchen Bei+5

University of Illinois Urbana-Champaign

Long-Context Inference Long-Context Reasoning in Language Models

Open paper AskPDF arXiv

Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long-Context Reasoning (RECONTEXT), a training-free inference method for improving long-context reasoning. RECONTEXT uses model-internal relevance signals to construct a query-conditioned evidence pool and replays it before final generation while preserving the full original context. This recursive selection process separates evidence organization from answer generation without training, external memory, or context pruning. We also provide a theoretical analysis based on associative memory, which characterizes the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation. Experiments on eight long-context datasets with 128K context length show that RECONTEXT consistently improves evidence utilization across Qwen3-4B, Qwen3-8B, and Llama3-8B, achieving the best average rank on all three backbones. Code is available at https://github.com/Yanjun-Zhao/ReContext.

Abstract

LLM Context Management

Explore similar work

ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning

RecaLLM: Addressing the Lost-in-Thought Phenomenon with Explicit In-Context Retrieval

Emergence of Context Characteristics Sensitivity in Large Language Models