Can Computation from Earlier Problems Help LLMs Solve New Ones?

Jipei He; Wenhui Tan; Xiaoyi Yu; Enver Sangineto; Fiorenzo Parascandolo; Rita Cucchiara; Ruihua Song

cs.AISep 30, 2026

Can Computation from Earlier Problems Help LLMs Solve New Ones?

Authors: Jipei He, Wenhui Tan, Xiaoyi Yu, Enver Sangineto, Fiorenzo Parascandolo, Rita Cucchiara, Ruihua Song

Organizations: Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · University of Modena and Reggio Emilia, Italy

Abstract

Large language models often solve independent problems in the same conversation. Can computation from earlier problems help them solve new ones? To answer this question, we first conduct preliminary experiments showing that retained history can raise or lower later-turn accuracy, even within the same domain. To understand these effects, we use controlled replay to isolate internal state changes specific to each problem-history pairing. Across different histories, these changes preserve similar relationships among current problems. To improve reasoning under retained history, we introduce STAIR (Stale-Token Attention for Inter-query Reuse). STAIR captures keys and values from earlier response generation in a fixed bank. It learns to redirect current queries when they read this bank during prompt processing. The base model remains frozen; only 12,288 parameters are trained. Across three Qwen models and four benchmarks, STAIR improves average later-turn accuracy by up to 11.67 percentage points over the unmodified model with history.

Figures & tables

Figure 1: STAIR in a continuous problem-solving session. (a) Earlier problems leave conversation history; the assistant responses illustrate Instruct, where reasoning and answers remain visible. T4 introduces a new independent problem. (b) Native is the unmodified model reading that history. (c) STAIR changes how current queries read a separate bank of attention keys and values captured during earlier turns. It does so while processing the current prompt (prefill), with the model weights fixed. Both conditions process the visible conversation anew each turn. (d) Avg@4 averages correctness over four sampled answers per problem for the two 4B models. Vanilla uses T1; Native and STAIR entries average T2–T4.

\dagger

AMC23 excludes training-overlap problems, leaving 34 problems.

Figure 2: Historical computation after task switches in Qwen3-4B Instruct on MATH-500. A: Crossed-history replay with a frozen backbone. Each of two disjoint data groups crosses 128 current problems with 32 three-turn source histories. State displacement is decomposed into additive effects and an interaction response. B: Attention mass on earlier assistant-response bodies (left) and the norm of the current problem’s state displacement (right; logarithmic scale). C: Cross-history overlap of eight nearest candidate problems in interaction-response space, with the Question-identity shuffle control. At layer 19, interaction energy fractions are 0.69% and 0.66% for groups 1 and 2. In B and C, solid lines denote group 1; dashed lines with hollow squares denote group 2. Vertical dotted lines mark layer 19, reported in the text. Layers are zero-indexed. D: Native answer transitions relative to Vanilla T1, normalized by 2,000 pairs per turn. Wrong-to-correct contributions are positive and correct-to-wrong contributions negative. Diamonds mark net changes.

T1	$n$	Vanilla	Native	STAIR
Correct	18,879	93.96	93.42	94.18
Wrong	1,425	80.07	60.35	68.56

Table 1: Accuracy (%) pooled over T2–T4 across three models and three mathematical benchmarks;

n

counts responses.

Figure 3: STAIR within a controlled block. (a) The change between two reads of the historical bank passes through the frozen output path and is added to the native attention output. Qwen3.5 also uses its native gate. (b) A learned reflection redirects the current query after auxiliary position encoding (RoPE). The original and reflected queries read the same fixed bank; subtracting their readouts gives

\Delta r

. Equation 7 specifies the reflected read. Only the reflection directions are learned.

Qwen3-4B Instruct
	MATH-500		AIME 2025		AMC23 †		GPQA-Diamond
Condition	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4
Vanilla	95.00	97.20	47.50	66.67	92.65	97.06	58.59	78.79
Native	94.10	97.60	34.44	47.78	93.38	96.08	58.00	76.94
STAIR	94.63	97.53	38.06	52.22	93.38	97.06	58.75	77.78
Qwen3.5-4B

Table 2: Continuous reasoning results (%). Vanilla reports T1; Native and STAIR average T2–T4. Bold marks the better Native/STAIR score for each metric, including ties. Underlined STAIR scores exceed the corresponding Vanilla reference.

Condition	Avg@4	$\Delta$ (pp)	Pass@4
Native	21.67	0.00	56.67
Random-reflector	37.22	+15.56	63.33
Direct reflected-read	41.67	+20.00	73.33
Bank-free	36.67	+15.00	66.67
K–V pairing permutation	33.06	+11.39	61.11
STAIR	48.33	+26.67	80.00

Table 3: Shared-T1 AIME 2025 results for Qwen3.5-4B at T2 (%). Conditions share 30 problems and four matched trajectories per problem. Random-reflector and K–V pairing average three test-time seeds; other conditions use one evaluation.

\Delta

is the Avg@4 difference from Native.

		Avg@4		Pass@4
Condition	Trainable params	Score	$\Delta$ (pp)	Score	$\Delta$ (pp)
Native	0	34.44	–	47.78	–
STAIR	12,288	38.06	+3.61	52.22	+4.44
LoRA	4,128,768	39.44	–	58.89	–
LoRA + STAIR	4,141,056	43.33	+3.89	63.33	+4.44

Table 4: Qwen3-4B Instruct on AIME 2025 (%). Scores average T2–T4.

\Delta

is computed before rounding, relative to Native for STAIR and to LoRA for LoRA + STAIR. Bold marks the highest score and gain for each metric, including ties.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Design	Group	Current problems	Histories	Sessions	Source problems
History coverage	1	64	128	33	81
History coverage	2	64	128	32	83
Problem coverage	1	128	32	25	67
Problem coverage	2	128	32	23	63

Appendix

Table 5: Balanced replay designs. Histories are sampled paths; sessions count distinct problem orders. Source problems count distinct problems in the first three turns.

Figure 4: Native attention across source categories in the three-turn problem-coverage design. The four panels use common scales. Solid lines denote data group 1; dashed lines with hollow squares denote data group 2. Layer indices are zero-based.

Figure 5: Cross-history neighborhood overlap at layer 19 under three-turn histories. Original coordinates use the full-dimensional interaction response; shared whitening uses projected coordinates with a common transform. Purple compares matched problem identities; gray shows the Question-identity shuffle control. Lines connect the three evaluated neighborhood sizes.

Turn	W to C	C to W	C to C	W to W	Net change
T2	37	47	1,853	63	$-0.50$
T3	40	59	1,841	60	$-0.95$
T4	35	60	1,840	65	$-1.25$

Appendix

Table 6: Qwen3-4B Instruct on MATH-500: answer transitions relative to Vanilla T1. C and W denote correct and wrong answers. Net changes are in percentage points.

MATH-500
Turn	T1 answer	$n$	Vanilla	Native	STAIR
T2	Correct	1,900	1,808 (95.16)	1,798 (94.63)	1,806 (95.05)
T2	Wrong	100	92 (92.00)	92 (92.00)	94 (94.00)
T3	Correct	1,900	1,805 (95.00)	1,787 (94.05)	1,805 (95.00)
T3	Wrong	100	95 (95.00)	94 (94.00)	94 (94.00)
T4	Correct	1,900	1,804 (94.95)	1,778 (93.58)	1,784 (93.89)

Appendix

Table 7: Qwen3-4B Instruct: current-answer correct counts and conditional accuracy (%) by actual T1 correctness. Bold compares Native and STAIR within each group, including ties.

MATH-500
Turn	T1 answer	$n$	Vanilla	Native	STAIR
T2	Correct	1,920	1,848 (96.25)	1,768 (92.08)	1,813 (94.43)
T2	Wrong	80	72 (90.00)	42 (52.50)	59 (73.75)
T3	Correct	1,920	1,844 (96.04)	1,864 (97.08)	1,875 (97.66)
T3	Wrong	80	76 (95.00)	67 (83.75)	74 (92.50)
T4	Correct	1,920	1,844 (96.04)	1,873 (97.55)	1,878 (97.81)

Appendix

Table 8: Qwen3.5-4B: current-answer correct counts and conditional accuracy (%) by actual T1 correctness. Bold compares Native and STAIR within each group, including ties.

MATH-500
Turn	T1 answer	$n$	Vanilla	Native	STAIR
T2	Correct	1,933	1,874 (96.95)	1,850 (95.71)	1,857 (96.07)
T2	Wrong	67	59 (88.06)	36 (53.73)	46 (68.66)
T3	Correct	1,933	1,867 (96.59)	1,892 (97.88)	1,884 (97.47)
T3	Wrong	67	66 (98.51)	54 (80.60)	58 (86.57)
T4	Correct	1,933	1,868 (96.64)	1,892 (97.88)	1,900 (98.29)

Appendix

Table 9: Qwen3.5-9B: current-answer correct counts and conditional accuracy (%) by actual T1 correctness. Bold compares Native and STAIR within each group, including ties.

Figure 6: Historical reads and next-layer MLP prediction. A: Incremental test

R^{2}

from adding the earlier-assistant read at four measured layers. B: Point estimates for equal-width feature additions at layer 3. Shuffling permutes assistant reads across histories within a problem, or across problems within a history, separately in fit and test groups. Colors and markers identify the two data-group exchange directions.

Figure 7: Training scope and bank accumulation. (a) The auxiliary branch acts on the current prompt suffix; response losses propagate through the current prefix computation. (b) T1 provides history, and T2–T4 receive response supervision. Each

\mathcal{B}_{t}

contains detached states from all preceding turns. New entries come from the turn’s replay and are appended for its successor.

Model	Query heads/layer	$d_{h}$	Init. std.	Parameters
Qwen3-4B Instruct	32	128	$1/\sqrt{128}$	12,288
Qwen3.5-4B	16	256	$1/16$	12,288
Qwen3.5-9B	16	256	$1/16$	12,288

Appendix

Table 10: Controller configuration. All query heads are controlled at layers

[3,11,19]

(zero-based). Normals are separate for each layer and head.

Model	Temp.	Top- $p$	Top- $k$	Presence penalty	Output limit
Qwen3-4B Instruct	0.7	0.8	20	0	16,384
Qwen3.5-4B	1.0	0.95	20	1.5	81,920 / 32,768
Qwen3.5-9B	1.0	0.95	20	1.5	81,920 / 32,768

Appendix

Table 11: Recorded decoding settings. Output limits count newly generated tokens per problem.

Readable bank	Truncated	Avg@4	Pass@4
Native	–	21.67	56.67
STAIR 2K	120/120	35.00	73.33
STAIR 8K	114/120	43.33	80.00
STAIR 32K	81/120	39.17	70.00
STAIR Full	0/120	48.33	80.00

Appendix

Table 12: Bank-prefix accuracy on shared-T1 AIME 2025 at T2 (%). Each condition uses four matched sample seeds per problem. Truncated counts banks longer than the access budget.

Bank	K/V	Transfer	Prefill	Decode	Peak	Total
	(MiB)	(ms)	(ms)	(s)	(GiB)	(s)
Native	0	$\approx 0$	70	10.08	8.03	10.15
2K	24	6	110	10.02	8.19	10.15
8K	96	24	157	10.06	8.70	10.23
32K	384	138	338	10.05	10.71	10.56
Full	$\approx 946$	330	628	9.98	14.61	10.95

Appendix

Table 13: Bank-size and fixed-workload costs across 16 histories. Transfer moves the bank to the GPU; total includes transfer, current-question prefill, and a fixed 256-token decode. Entries summarize each history before taking the median across histories.

MATH-500
	T1		T2		T3		T4		Mean
Condition	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4
Vanilla	95.00	97.20	–	–	–	–	–	–	–	–
Native	95.00	97.20	94.50	97.40	94.05	97.60	93.75	97.80	94.10	97.60
STAIR	95.00	97.20	95.00	97.60	94.95	97.60	93.95	97.40	94.63	97.53
AIME 2025

Appendix

Table 14: Qwen3-4B Instruct: results by turn (%). Mean averages T2–T4.

	T2		T3		T4		Mean
Condition	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4
Native	36.67	50.00	32.50	40.00	34.17	53.33	34.44	47.78
STAIR	40.00	53.33	35.83	50.00	38.33	53.33	38.06	52.22
LoRA	40.83	53.33	38.33	60.00	39.17	63.33	39.44	58.89
LoRA + STAIR	44.17	63.33	45.00	70.00	40.83	56.67	43.33	63.33

Appendix

Table 15: Qwen3-4B Instruct on AIME 2025: standalone and jointly trained adapters by turn (%). Mean averages T2–T4; bold marks the highest score in each column, including ties.

MATH-500
	T1		T2		T3		T4		Mean
Condition	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4
Vanilla	96.00	98.60	–	–	–	–	–	–	–	–
Native	96.00	98.60	90.50	98.20	96.55	99.20	97.30	99.00	94.78	98.80
STAIR	96.00	98.60	93.60	99.40	97.45	99.40	97.75	99.00	96.27	99.27
AIME 2025

Appendix

Table 16: Qwen3.5-4B: results by turn (%). Mean averages T2–T4.

MATH-500
	T1		T2		T3		T4		Mean
Condition	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4	Avg@4	Pass@4
Vanilla	96.65	98.20	–	–	–	–	–	–	–	–
Native	96.65	98.20	94.30	98.80	97.30	99.60	97.55	99.20	96.38	99.20
STAIR	96.65	98.20	95.15	98.80	97.10	99.40	98.10	99.20	96.78	99.13
AIME 2025

Appendix

Table 17: Qwen3.5-9B: results by turn (%). Mean averages T2–T4.

Explore similar work

Stateful Reasoning via Insight Replay
May 14, 2026Bin Lei, Caiwen Ding, Jiachen Yang +2Chain-of-Thought Reasoning Deepseek
University of Minnesota · 2Simular AI
Multi-Turn Reasoning When Context Arrives in Pieces: Scalable Sharding and Memory-Augmented RL
Jun 11, 2026Shu Tong Luo, Wenqin Liu, Rui Liu +2Long-Context Reasoning Shard
The University of Melbourne · The University of Tokyo
Large Language Models Decide Early and Explain Later
Apr 24, 2026Ayan Datta, Zhixue Zhao, Bhuvanesh Verma +3LLM Reasoning Strategies Chain-of-Thought Reasoning
IIIT Hyderabad · University of Sheffield, UK · Goethe University, Frankfurt am Main, Germany

May 14, 2026, cs.AIJ/K move · Enter open · S save

Stateful Reasoning via Insight Replay

Bin Lei, Caiwen Ding, Jiachen Yang, Ang Li+1

University of Minnesota · 2Simular AI

Chain-of-Thought Reasoning Deepseek Replay

Open paper AskPDF arXiv

Chain-of-Thought (CoT) reasoning has become a foundation for eliciting multi-step reasoning in large language models, but recent studies show that its benefits do not scale monotonically with chain length: while longer CoT generally enables a model to tackle harder problems, on a given problem, accuracy typically increases with CoT length up to a point, after which it declines. We identify a major cause of this phenomenon: as the CoT grows, the model's attention to critical insights produced earlier in the trace gradually weakens, making those insights progressively less accessible when they are most needed. Therefore, we propose \textbf{InsightReplay}, a stateful reasoning approach in which the model periodically extracts critical insights from its reasoning trace and replays them near the active generation frontier, keeping them accessible as the reasoning scales. Extensive experiments on a

\mathbf{2}\!\times\!\mathbf{3}\!\times\!\mathbf{4}

benchmark grid, covering model scales

\{\text{8B}, \text{30B}\}

, model families

\{\text{Qwen3.5}, \text{DeepSeek-R1-Distill-Qwen}, \text{Gemma-4}\}

, and reasoning benchmarks

\{\text{AIME}, \text{HMMT}, \text{GPQA Diamond}, \text{LiveCodeBench v5}\}

, show that 3-round InsightReplay yields accuracy gains across \textbf{all 24 settings}, with an averaged improvement of

\mathbf{+1.65}

points over standard CoT, and a largest single-setting gain of

\mathbf{+9.2}

points on R1-Distill-32B's LiveCodeBench v5 subset. Our results suggest that the effectiveness of test-time scaling depends not only on how much a model reasons, but also on whether critical intermediate insights remain accessible throughout long reasoning trajectories.

Abstract

Explore similar work

Stateful Reasoning via Insight Replay

Multi-Turn Reasoning When Context Arrives in Pieces: Scalable Sharding and Memory-Augmented RL

Large Language Models Decide Early and Explain Later