Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction

Jiwon Jeon; Myungsik Cho; Jeonghye Kim; Youngchul Sung

cs.CLOct 9, 2026

Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction

Authors: Jiwon Jeon, Myungsik Cho, Jeonghye Kim, Youngchul Sung

Organizations: Korea Advanced Institute of Science and Technology (KAIST)

Abstract

Large language models struggle to reliably identify and correct errors in their own reasoning. Unlike prior training-based approaches that fine-tune an LLM to generate revisions or feedback, we learn where to intervene in failed rollouts and determine how to redirect them. To choose where to intervene, we train a 130K-parameter MLP head on a frozen encoder using only final-answer correctness. For redirection, we find that under a fixed generation budget, the tested inline injections fail to reliably redirect reasoning, whereas presenting the same instruction in a new user turn changes the model's course. The resulting correction framework, UPTURN (User-Prompt TURN injection at a learned position), requires no LLM fine-tuning: it retains a selected prefix, injects an instruction as a new user turn, and regenerates the suffix. Across four reasoners and six math benchmarks, UPTURN improves average accuracy in every setting over a 32-rollout baseline. It outperforms self-correction and reflection-token injection baselines under matched correction budgets, transfers across reasoners and to GPQA-Diamond without retraining, and complements GRPO and self-correction methods.

Figures & tables

Panel of Figure 1 — Figure 1: Structure, not content alone, enables redirection. (Left) The same instruction is absorbed inline but redirects reasoning in a new user turn. (Right) Learning where to intervene raises rescue rate (the fraction of failed rollouts corrected; top) and six-benchmark mean accuracy (bottom).

Panel of Figure 2 — Figure 2: Why turn-boundary injection is needed. (a) Prefix value along failed rollouts, computed as the mean success rate over 16 continuations sampled from each prefix. (b) Position-wise rescue gap $R{-}V$ for inline (in) and multi-turn (mt) injection, where $R$ is post-injection success and $V$ is prefix value. (c) Oracle best rescue rate obtained by selecting, for each problem and phrase, the position with highest post-injection success.

Figure 3: Overview of UPTURN. (Step 1) A frozen 0.6B encoder and MLP head

f_{\phi}

score paragraph boundaries ( \n\n ) in failed rollouts ( ✗ ); the policy selects a cut

t^{\star}

. (Step 2) UPTURN retains the prefix, inserts a fixed instruction as a new user turn, and regenerates in a new assistant turn, recovering a correct answer ( ✓ ).

Method	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
Base@32	32.6	63.3	21.8	46.7	26.7	43.3	67.0	92.5	19.2	27.3	85.0	93.2
UPTURN@16	50.8	83.3	39.0	56.7	44.2	70.0	84.2	95.0	28.0	39.4	91.0	95.8
$\Delta$	+18.2	+20.0	+17.2	+10.0	+17.5	+26.7	+17.2	+2.5	+8.8	+12.1	+6.0	+2.6
Qwen3-8B

Table 1: Results on six math benchmarks. Base@32 uses 32 rollouts; UPTURN@16 uses 16 rollouts and corrects each gate-flagged one once.

\Delta

denotes UPTURN’s gain over Base; bold marks the higher score.

Budget	Reflexion@16	SCoRe@16	AlphaOne@16	CyclicReflex@16	UPTURN@16
	Self-correction		Reflection-token injection
$L_{\max}$	43.6 / 61.4	– †	32.6 / 45.2	35.7 / 49.7	59.8 / 77.2 ( $+16.2$ / $+15.9$ )
$2L_{\max}$	55.3 / 77.5	52.3 / 79.9	55.0 / 71.8	57.2 / 73.5	62.2 / 80.7 ( $+4.9$ / $+0.8$ )

Table 2: Comparison with other baselines. Avg / Pass averaged over six benchmarks and all reasoners except QwQ-32B, at two budgets (

L_{\max}

and

2L_{\max}

). Parentheses give UPTURN’s gain over the best baseline at each budget; bold marks the best scores. † SCoRe requires

2L_{\max}

as it conditions on the full first attempt.

Panel of Figure 4 — Figure 4: Position-policy transfer. Averages over six benchmarks. (a) Avg gain over Base when the policy trained on the source reasoner is applied to the target; parentheses (and colour) give the difference from the target’s own policy on the diagonal. (b) UPTURN gains over Base and GRPO, averaged over four reasoners; the GRPO setting reuses the corresponding Base-trained position policy without retraining.

Reasoner	Base@32		UPTURN@16		$\boldsymbol{\Delta}$
Reasoner	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B	32.5	56.6	50.4	79.8	+17.9	+23.2
Qwen3-8B	35.0	61.1	54.3	83.8	+19.3	+22.7
DS-R1	20.7	49.0	35.9	78.8	+15.2	+29.8
QwQ-32B	41.5	68.7	54.2	80.8	+12.7	+12.1

Table 3: Transfer to GPQA-Diamond .

\Delta

denotes UPTURN’s gain over Base; bold marks the higher score. DS-R1 abbreviates DeepSeek-R1-Qwen3-8B.

Avg / Pass	Reflexion@16	SCoRe@16
Baseline	55.3 / 77.5	52.3 / 79.9
+ UPTURN	64.1 / 80.5	64.1 / 81.8
$\Delta$	+8.8 / +3.0	+11.8 / +1.8

Table 4: UPTURN after self-correction. Avg / Pass averaged over six benchmarks and three reasoners (excluding QwQ-32B), at

2L_{\max}

\Delta

gives the added gain; bold marks the better score in each pair.

Panel of Figure 5 — Figure 5: Position and injection ablations. Avg and Pass gains over Base@32, averaged over six benchmarks. (a) Fresh regeneration and injection at start, random, or learned positions, using the same gate and replacement rule; averages over four reasoners. (b) Inline and multi-turn injection with separately trained position policies.

Method	Avg	GS	Pass
Base@32	37.1	52.0	53.5
UPTURN@16	55.3	63.0	71.3
$\Delta$	+18.2	+11.0	+17.7
UPTURN@16	55.3	63.0	71.3
UPTURN@16+GT	55.9	63.3	71.3
$\Delta$	+0.6	+0.2	+0.1

Table 5: Gate ablations , averaged over four reasoners and six benchmarks. GS : accuracy of the highest-gate candidate. +GT: flagging with a ground-truth gate.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: One failed Qwen3-4B (Thinking on) rollout continued three ways from the same cut. (A) Unmodified continuation remains trapped in re-verification. (B) Inline injection is absorbed as the model’s own reasoning and resumes the loop. (C) The same instruction in a new user turn redirects reasoning to the correct answer. Full rollouts are provided in Appendix .

Hyperparameter	Value
Batch size $B$ (prompts with $\geq$ 1 failure)	64
Rollouts per prompt $N$	8
Cut positions per rollout $G$	8
Continuations per position $M$	8
Generation budget $L_{\max}$	8192
Max generation batches (dynamic sampling)	8

Appendix

Table 6: Hyperparameters

Family (reasoners)	Injected sequence at the cut
Qwen ChatML (Qwen3-4B/8B, QwQ-32B)	<\|im_end\|>\n<\|im_start\|>user\n $c$ <\|im_end\|>\n<\|im_start\|>assistant\n
DeepSeek (DeepSeek-R1-0528-Qwen3-8B)	<\|end_of_sentence\|><\|User\|> $c$ <\|Assistant\|><think>\n

Appendix

Table 7: Turn-boundary injection sequence per model family.

c

denotes the fixed instruction.

Benchmark	AIME24	AIME25	AIME26	AMC23	HMMT	MATH500	Avg
AUC	.982	.983	.980	.984	.920	.922	.962

Appendix

Table 8: Gate AUC by benchmark. Mean over four reasoners.

Model	Avg			Pass
Model	Base@32	UPTURN@16	UPTURN@16+GT	Base@32	UPTURN@16	UPTURN@16+GT
Qwen3-4B	42.1	56.2	56.6	61.1	73.4	73.5
Qwen3-8B	41.4	58.3	58.7	60.5	72.7	72.8
DeepSeek-R1-Qwen3-8B	19.2	49.5	50.9	28.8	70.3	70.3
QwQ-32B	45.7	57.2	57.4	63.8	68.7	68.7
Model Avg	37.1	55.3	55.9	53.5	71.3	71.3

Appendix

Table 9: Learned (UPTURN@16) versus ground-truth (+GT) gating , averaged over six benchmarks. Bold marks the higher score; ties are both bold.

$\gamma$	0	0.1	0.2	0.4	0.6	0.8	1.0
Avg	37.3	55.6	55.5	55.3	55.2	55.2	55.2
Pass	51.8	71.2	71.3	71.3	71.3	71.3	71.3
Corrected (%)	0	64	69	79	91	99	100

Appendix

Table 10: Sensitivity to the gate threshold

\gamma

. Avg and Pass over four reasoners and six benchmarks; Corrected is the fraction of rollouts that receive a correction. Bold marks the default. At

\gamma{=}0

nothing is corrected.

Method	AIME24			AIME25			AIME26
	Avg	GS	Pass	Avg	GS	Pass	Avg	GS	Pass
Qwen3-4B
Base@32	32.6	60.0	63.3	21.8	46.7	46.7	26.7	40.0	43.3
UPTURN@16	50.8	66.7	83.3	39.0	53.3	56.7	44.2	53.3	70.0
$\Delta$	+18.2	+6.7	+20.0	+17.2	+6.7	+10.0	+17.5	+13.3	+26.7
Qwen3-8B

Appendix

Table 11: Gate as a selector , per benchmark. GS is the accuracy of the highest-gate candidate; Avg and Pass are as in Table 1 .

\Delta

denotes UPTURN’s gain over Base; bold marks the higher score.

Method	Gens./q	Gen. tokens/q	Wall-clock (s)	vs. Base@32
Base@16	16.0	122,881	$477\pm 2$	$0.51\times$
Base@32	32.0	245,548	$931\pm 0$	$1.00\times$
UPTURN@16	28.6	156,619	$\mathbf{790\pm 2}$	$\mathbf{0.85\times}$

Appendix

Table 12: Wall-clock cost on AIME24 (Qwen3-4B, one B200).

Figure 7: Avg and Pass against the number of correction rounds , averaged over six benchmarks. Colour marks the reasoner; solid lines are gated , dashed lines naive .

Figure 8: Avg and rescue rate under varied injection instructions , averaged over six benchmarks. Solid bars are Avg, hatched bars rescue rate; Base has no rescue rate. Instructions are ordered by mean Avg; None inserts the turn boundary with no text.

	Boundary entropy		Other entropy		Top-5%	Boundary
Reasoner	mean	median	mean	median	coverage	frac.
Qwen3-4B	0.62	0.61	0.29	0.008	18%	2.0%
Qwen3-8B	0.60	0.59	0.27	0.004	19%	2.0%
DeepSeek-R1-Qwen3-8B	0.99	1.01	0.31	0.018	35%	3.2%
QwQ-32B	0.81	0.80	0.41	0.095	22%	2.6%

Appendix

Table 13: Next-token entropy at \n\n boundaries vs. elsewhere on incorrect AIME24/25 rollouts; coverage is the share of top-5% entropy positions within

\pm 2

tokens of a boundary.

Figure 9: Scaling the generation budget. Avg (top) and Pass (bottom) averaged over six benchmarks as the per-rollout budget grows from

L_{\max}

3L_{\max}

; labels give UPTURN’s gain over Base at each budget.

Method	Budget	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
		Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
Reflexion@16	$L_{\max}$	51.5	76.7	36.0	63.3	42.5	66.7	83.9	92.5	25.6	33.3	91.1	96.6
UPTURN@16	$L_{\max}$	59.0	83.3	48.3	70.0	53.8	76.7	88.4	97.5	33.9	45.5	92.1	96.4
$\Delta$ (vs. best)		+7.5	+6.7	+12.3	+6.7	+11.2	+10.0	+4.5	+5.0	+8.3	+12.1	+1.0	-0.2
Reflexion@16	$2L_{\max}$	64.2	86.7	49.0	76.7	55.6	80.0	90.2	100.0	34.3	57.6	93.4	97.8

Appendix

Table 14: Full comparison with self-correction baselines , per reasoner and benchmark, at

L_{\max}

and

2L_{\max}

correction budgets.

\Delta

rows give UPTURN’s gain over the best baseline at the same budget; bold marks the best entry per column and budget. SCoRe conditions on the full first attempt, so it is evaluated at

2L_{\max}

only.

Method	Budget	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
		Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
AlphaOne@16	$L_{\max}$	33.8	53.3	21.9	36.7	27.3	46.7	67.0	87.5	16.9	24.2	84.9	92.4
CyclicReflex@16	$L_{\max}$	37.7	56.7	25.2	50.0	28.8	46.7	70.2	92.5	19.1	27.3	86.1	92.6
UPTURN@16	$L_{\max}$	59.0	83.3	48.3	70.0	53.8	76.7	88.4	97.5	33.9	45.5	92.1	96.4
$\Delta$ (vs. best)		+21.2	+26.7	+23.1	+20.0	+25.0	+30.0	+18.3	+5.0	+14.8	+18.2	+6.1	+3.8

Appendix

Table 15: Full comparison with reflection-token injection , per reasoner and benchmark, at

L_{\max}

and

2L_{\max}

correction budgets.

\Delta

rows give UPTURN’s gain over the best injection method at the same budget; bold marks the best entry per column and budget.

Method	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
Reflexion@16 ( $L_{\max}$ )	51.5	76.7	36.0	63.3	42.5	66.7	83.9	92.5	25.6	33.3	91.1	96.6
+ UPTURN	49.0	73.3	35.8	53.3	42.5	60.0	84.4	92.5	23.7	33.3	90.9	97.0
$\Delta$	-2.5	-3.3	-0.2	-10.0	+0.0	-6.7	+0.5	+0.0	-1.9	+0.0	-0.3	+0.4
Reflexion@16 ( $2L_{\max}$ )	64.2	86.7	49.0	76.7	55.6	80.0	90.2	100.0	34.3	57.6	93.4	97.8

Appendix

Table 16: UPTURN on top of a self-correction baseline , per reasoner and benchmark. Each baseline produces its own 16 corrected solutions; UPTURN’s gate then flags the weak ones among them and redirects each once, at the budget named on the baseline row. Reflexion is shown at both

L_{\max}

and

2L_{\max}

; SCoRe conditions on the full first attempt, so only

2L_{\max}

is meaningful for it.

\Delta

rows give the gain of adding UPTURN; bold marks the better entry of the pair.

Method	AIME24		AIME25		AIME26
	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
Base@32	33.4 $\pm$ 0.8	62.2 $\pm$ 1.9	21.6 $\pm$ 0.2	47.8 $\pm$ 1.9	26.1 $\pm$ 1.2	43.3 $\pm$ 0.0
UPTURN@16	52.1 $\pm$ 1.1	81.1 $\pm$ 3.8	40.0 $\pm$ 1.0	58.9 $\pm$ 1.9	45.8 $\pm$ 1.5	70.0 $\pm$ 0.0
$\Delta$	+18.7	+18.9	+18.4	+11.1	+19.7	+26.7
Qwen3-8B

Appendix

Table 17: Seed robustness. Table 1 repeated with three seeds for training the position policy and drawing rollouts; mean

\pm

std over the three runs.

\Delta

is UPTURN’s gain; bold marks the higher mean.

Method	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
AceReason-Nemotron-1.1-7B
Base@32	37.2	73.3	27.7	40.0	31.5	56.7	79.7	95.0	19.5	30.3	88.4	96.2
UPTURN@16	47.5	73.3	32.5	46.7	44.6	63.3	89.7	95.0	27.1	45.5	92.1	97.0
$\Delta$	+10.3	0.0	+4.8	+6.7	+13.1	+6.7	+10.0	0.0	+7.6	+15.2	+3.7	+0.8
Skywork-OR1-7B

Appendix

Table 18: Results on two additional reasoners , with the same setup as Table 1 .

\Delta

denotes UPTURN’s gain over Base; bold marks improvements.

LLM Self-Correction

Large language model self-correction studies how language models detect and repair errors in their own answers, reasoning, and actions.

See all 159 papers

Jun 4, 2026

The Self-Correction Illusion: LLMs Correct Others but Not Themselves

Recent work shows that LLM agents struggle to correct errors in their own reasoning traces yet show markedly higher correction rates when identical claims appear under external sources. We ask whether this asymmetry reflects a capability deficit or a role-label artifact: does an agent's willingness to correct a wrong claim depend causally on the chat-template role that carries it, rather than on the claim's content? Our setup keeps the erroneous claim byte-identical across all conditions (SHA-256 verified) and varies only its wrapping role: the agent's own \role{<thought>}, a \role{user} message, a \role{tool} response, or a \role{system <memory>} block. Across 13 model-domain cells covering seven model families and three domains (

n{=}30

paired tasks per cell), relabeling the claim from \role{<thought>} to an external role lifts the explicit-correction rate by 23 to 93 percentage points, with 10 of 13 cells reaching

p{<}0.001

. Further experiments confirm that the effect is asymmetric, mechanistically decomposable, and robust across domains. The failure to self-correct is not a cognitive deficit; it is a chat-template artifact. We exploit this artifact by designing a prompt-structure-only intervention that requires no training and no model modification, with its strongest role label being domain-dependent: \role{<memory>} dominates on math, while a plain \role{user} message dominates on logical deduction.

Kuan-Yen Chen, Fang-Yi Su, Jung-Hsien Chiang

National Cheng Kung University

LLM Self-Correction LLM Prompting Causal Interventions in Language Models LLM Reasoning

Ask PDF

Oct 1, 2025

In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration

LLM-generated drafts often contain subtle factual or logical errors, yet prior work shows that models struggle to reliably integrate multi-turn feedback aimed at fixing them. We propose in-place feedback, an interaction paradigm in which the user directly edits the model's previous response and the model continues generation from the edited context. In-place feedback consistently outperforms standard multi-turn feedback across five reasoning-intensive benchmarks while requiring fewer tokens, and our fine-grained analysis shows that it applies corrections more reliably and propagates them to subsequent reasoning. A user study with domain experts refining LLM-generated summaries corroborates these findings: participants report higher final-output satisfaction and substantially lower fatigue with in-place feedback, and a mixed strategy combining in-place and multi-turn feedback improves every dimension we measure. These results suggest that editing errors in place is a more reliable way to incorporate expert corrections than multi-turn feedback, while combining the two acts complementarily to form a more effective collaboration than either mechanism alone.

Youngbin Choi, Minjong Lee, Saemi Moon +4

Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science and Engineering, POSTECH

LLM Reliability Multi-Turn Language Model Reasoning LLM Self-Refinement

Ask PDF

May 29, 2026

What Am I Missing? Question-Answering as Hidden State Probing

Test-time reasoning has become a significant field of study since the introduction of chain-of-thought reasoning in large language models (LLMs). However, the mechanisms of this reasoning process are still under-explored -- from the same input prompt, and even the same partial solution, LLMs can produce varied answers if sampled multiple times. We propose to leverage question-asking as an inference-time intervention that articulates information about the model's hidden state. To achieve that, we present a student-teacher setting where a student asks questions to a teacher. We train a probe on the student's hidden state before and after asking a question and find it is predictive of the trajectory's final correctness, even before generating the teacher's answer. This suggests there is a meaningful signal from the self-diagnosis that occurs during question generation rather than information transfer from the teacher. We then frame question-asking as a sequential decision problem, using this probe as a quality score, and define a gating policy to ask questions that maximize likelihood of correctness. We find that the success of question-asking as an intervention is largely dependent on the model's self-consistency. Our empirical results show a gap between detection and recovery; while our gating policy captures model correctness and uncertainty, interventions are equally likely to harm correct trajectories as they are to recover incorrect ones. This gap between diagnosis and correction has broader implications on language models' capacity for self-refinement under uncertainty.

Chu Fei Luo, Samuel Dahan, Xiaodan Zhu

Department of Electrical and Computer Engineering & Ingenuity Labs, Queen’s University · Conflict Analytics Lab, Queen’s University · Cornell Law School +1

LLM Self-Refinement Language Model Self-Assessment LLM Uncertainty Estimation Metacognition in Language Models Language Model Probing LLM Reasoning

Ask PDF

Abstract

LLM Self-Correction

Explore similar work

The Self-Correction Illusion: LLMs Correct Others but Not Themselves

In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration

What Am I Missing? Question-Answering as Hidden State Probing