Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction

Jiwon Jeon; Myungsik Cho; Jeonghye Kim; Youngchul Sung

cs.CLOct 9, 2026

Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction

Authors: Jiwon Jeon, Myungsik Cho, Jeonghye Kim, Youngchul Sung

Organizations: Korea Advanced Institute of Science and Technology (KAIST)

Abstract

Large language models struggle to reliably identify and correct errors in their own reasoning. Unlike prior training-based approaches that fine-tune an LLM to generate revisions or feedback, we learn where to intervene in failed rollouts and determine how to redirect them. To choose where to intervene, we train a 130K-parameter MLP head on a frozen encoder using only final-answer correctness. For redirection, we find that under a fixed generation budget, the tested inline injections fail to reliably redirect reasoning, whereas presenting the same instruction in a new user turn changes the model's course. The resulting correction framework, UPTURN (User-Prompt TURN injection at a learned position), requires no LLM fine-tuning: it retains a selected prefix, injects an instruction as a new user turn, and regenerates the suffix. Across four reasoners and six math benchmarks, UPTURN improves average accuracy in every setting over a 32-rollout baseline. It outperforms self-correction and reflection-token injection baselines under matched correction budgets, transfers across reasoners and to GPQA-Diamond without retraining, and complements GRPO and self-correction methods.

Figures & tables

Panel of Figure 1 — Figure 1: Structure, not content alone, enables redirection. (Left) The same instruction is absorbed inline but redirects reasoning in a new user turn. (Right) Learning where to intervene raises rescue rate (the fraction of failed rollouts corrected; top) and six-benchmark mean accuracy (bottom).

Panel of Figure 2 — Figure 2: Why turn-boundary injection is needed. (a) Prefix value along failed rollouts, computed as the mean success rate over 16 continuations sampled from each prefix. (b) Position-wise rescue gap $R{-}V$ for inline (in) and multi-turn (mt) injection, where $R$ is post-injection success and $V$ is prefix value. (c) Oracle best rescue rate obtained by selecting, for each problem and phrase, the position with highest post-injection success.

Figure 3: Overview of UPTURN. (Step 1) A frozen 0.6B encoder and MLP head

f_{\phi}

score paragraph boundaries ( \n\n ) in failed rollouts ( ✗ ); the policy selects a cut

t^{\star}

. (Step 2) UPTURN retains the prefix, inserts a fixed instruction as a new user turn, and regenerates in a new assistant turn, recovering a correct answer ( ✓ ).

Method	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
Base@32	32.6	63.3	21.8	46.7	26.7	43.3	67.0	92.5	19.2	27.3	85.0	93.2
UPTURN@16	50.8	83.3	39.0	56.7	44.2	70.0	84.2	95.0	28.0	39.4	91.0	95.8
$\Delta$	+18.2	+20.0	+17.2	+10.0	+17.5	+26.7	+17.2	+2.5	+8.8	+12.1	+6.0	+2.6
Qwen3-8B

Table 1: Results on six math benchmarks. Base@32 uses 32 rollouts; UPTURN@16 uses 16 rollouts and corrects each gate-flagged one once.

\Delta

denotes UPTURN’s gain over Base; bold marks the higher score.

Budget	Reflexion@16	SCoRe@16	AlphaOne@16	CyclicReflex@16	UPTURN@16
	Self-correction		Reflection-token injection
$L_{\max}$	43.6 / 61.4	– †	32.6 / 45.2	35.7 / 49.7	59.8 / 77.2 ( $+16.2$ / $+15.9$ )
$2L_{\max}$	55.3 / 77.5	52.3 / 79.9	55.0 / 71.8	57.2 / 73.5	62.2 / 80.7 ( $+4.9$ / $+0.8$ )

Table 2: Comparison with other baselines. Avg / Pass averaged over six benchmarks and all reasoners except QwQ-32B, at two budgets (

L_{\max}

and

2L_{\max}

). Parentheses give UPTURN’s gain over the best baseline at each budget; bold marks the best scores. † SCoRe requires

2L_{\max}

as it conditions on the full first attempt.

Panel of Figure 4 — Figure 4: Position-policy transfer. Averages over six benchmarks. (a) Avg gain over Base when the policy trained on the source reasoner is applied to the target; parentheses (and colour) give the difference from the target’s own policy on the diagonal. (b) UPTURN gains over Base and GRPO, averaged over four reasoners; the GRPO setting reuses the corresponding Base-trained position policy without retraining.

Reasoner	Base@32		UPTURN@16		$\boldsymbol{\Delta}$
Reasoner	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B	32.5	56.6	50.4	79.8	+17.9	+23.2
Qwen3-8B	35.0	61.1	54.3	83.8	+19.3	+22.7
DS-R1	20.7	49.0	35.9	78.8	+15.2	+29.8
QwQ-32B	41.5	68.7	54.2	80.8	+12.7	+12.1

Table 3: Transfer to GPQA-Diamond .

\Delta

denotes UPTURN’s gain over Base; bold marks the higher score. DS-R1 abbreviates DeepSeek-R1-Qwen3-8B.

Avg / Pass	Reflexion@16	SCoRe@16
Baseline	55.3 / 77.5	52.3 / 79.9
+ UPTURN	64.1 / 80.5	64.1 / 81.8
$\Delta$	+8.8 / +3.0	+11.8 / +1.8

Table 4: UPTURN after self-correction. Avg / Pass averaged over six benchmarks and three reasoners (excluding QwQ-32B), at

2L_{\max}

\Delta

gives the added gain; bold marks the better score in each pair.

Panel of Figure 5 — Figure 5: Position and injection ablations. Avg and Pass gains over Base@32, averaged over six benchmarks. (a) Fresh regeneration and injection at start, random, or learned positions, using the same gate and replacement rule; averages over four reasoners. (b) Inline and multi-turn injection with separately trained position policies.

Method	Avg	GS	Pass
Base@32	37.1	52.0	53.5
UPTURN@16	55.3	63.0	71.3
$\Delta$	+18.2	+11.0	+17.7
UPTURN@16	55.3	63.0	71.3
UPTURN@16+GT	55.9	63.3	71.3
$\Delta$	+0.6	+0.2	+0.1

Table 5: Gate ablations , averaged over four reasoners and six benchmarks. GS : accuracy of the highest-gate candidate. +GT: flagging with a ground-truth gate.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: One failed Qwen3-4B (Thinking on) rollout continued three ways from the same cut. (A) Unmodified continuation remains trapped in re-verification. (B) Inline injection is absorbed as the model’s own reasoning and resumes the loop. (C) The same instruction in a new user turn redirects reasoning to the correct answer. Full rollouts are provided in Appendix .

Hyperparameter	Value
Batch size $B$ (prompts with $\geq$ 1 failure)	64
Rollouts per prompt $N$	8
Cut positions per rollout $G$	8
Continuations per position $M$	8
Generation budget $L_{\max}$	8192
Max generation batches (dynamic sampling)	8

Appendix

Table 6: Hyperparameters

Family (reasoners)	Injected sequence at the cut
Qwen ChatML (Qwen3-4B/8B, QwQ-32B)	<\|im_end\|>\n<\|im_start\|>user\n $c$ <\|im_end\|>\n<\|im_start\|>assistant\n
DeepSeek (DeepSeek-R1-0528-Qwen3-8B)	<\|end_of_sentence\|><\|User\|> $c$ <\|Assistant\|><think>\n

Appendix

Table 7: Turn-boundary injection sequence per model family.

c

denotes the fixed instruction.

Benchmark	AIME24	AIME25	AIME26	AMC23	HMMT	MATH500	Avg
AUC	.982	.983	.980	.984	.920	.922	.962

Appendix

Table 8: Gate AUC by benchmark. Mean over four reasoners.

Model	Avg			Pass
Model	Base@32	UPTURN@16	UPTURN@16+GT	Base@32	UPTURN@16	UPTURN@16+GT
Qwen3-4B	42.1	56.2	56.6	61.1	73.4	73.5
Qwen3-8B	41.4	58.3	58.7	60.5	72.7	72.8
DeepSeek-R1-Qwen3-8B	19.2	49.5	50.9	28.8	70.3	70.3
QwQ-32B	45.7	57.2	57.4	63.8	68.7	68.7
Model Avg	37.1	55.3	55.9	53.5	71.3	71.3

Appendix

Table 9: Learned (UPTURN@16) versus ground-truth (+GT) gating , averaged over six benchmarks. Bold marks the higher score; ties are both bold.

$\gamma$	0	0.1	0.2	0.4	0.6	0.8	1.0
Avg	37.3	55.6	55.5	55.3	55.2	55.2	55.2
Pass	51.8	71.2	71.3	71.3	71.3	71.3	71.3
Corrected (%)	0	64	69	79	91	99	100

Appendix

Table 10: Sensitivity to the gate threshold

\gamma

. Avg and Pass over four reasoners and six benchmarks; Corrected is the fraction of rollouts that receive a correction. Bold marks the default. At

\gamma{=}0

nothing is corrected.

Method	AIME24			AIME25			AIME26
	Avg	GS	Pass	Avg	GS	Pass	Avg	GS	Pass
Qwen3-4B
Base@32	32.6	60.0	63.3	21.8	46.7	46.7	26.7	40.0	43.3
UPTURN@16	50.8	66.7	83.3	39.0	53.3	56.7	44.2	53.3	70.0
$\Delta$	+18.2	+6.7	+20.0	+17.2	+6.7	+10.0	+17.5	+13.3	+26.7
Qwen3-8B

Appendix

Table 11: Gate as a selector , per benchmark. GS is the accuracy of the highest-gate candidate; Avg and Pass are as in Table 1 .

\Delta

denotes UPTURN’s gain over Base; bold marks the higher score.

Method	Gens./q	Gen. tokens/q	Wall-clock (s)	vs. Base@32
Base@16	16.0	122,881	$477\pm 2$	$0.51\times$
Base@32	32.0	245,548	$931\pm 0$	$1.00\times$
UPTURN@16	28.6	156,619	$\mathbf{790\pm 2}$	$\mathbf{0.85\times}$

Appendix

Table 12: Wall-clock cost on AIME24 (Qwen3-4B, one B200).

Figure 7: Avg and Pass against the number of correction rounds , averaged over six benchmarks. Colour marks the reasoner; solid lines are gated , dashed lines naive .

Figure 8: Avg and rescue rate under varied injection instructions , averaged over six benchmarks. Solid bars are Avg, hatched bars rescue rate; Base has no rescue rate. Instructions are ordered by mean Avg; None inserts the turn boundary with no text.

	Boundary entropy		Other entropy		Top-5%	Boundary
Reasoner	mean	median	mean	median	coverage	frac.
Qwen3-4B	0.62	0.61	0.29	0.008	18%	2.0%
Qwen3-8B	0.60	0.59	0.27	0.004	19%	2.0%
DeepSeek-R1-Qwen3-8B	0.99	1.01	0.31	0.018	35%	3.2%
QwQ-32B	0.81	0.80	0.41	0.095	22%	2.6%

Appendix

Table 13: Next-token entropy at \n\n boundaries vs. elsewhere on incorrect AIME24/25 rollouts; coverage is the share of top-5% entropy positions within

\pm 2

tokens of a boundary.

Figure 9: Scaling the generation budget. Avg (top) and Pass (bottom) averaged over six benchmarks as the per-rollout budget grows from

L_{\max}

3L_{\max}

; labels give UPTURN’s gain over Base at each budget.

Method	Budget	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
		Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
Reflexion@16	$L_{\max}$	51.5	76.7	36.0	63.3	42.5	66.7	83.9	92.5	25.6	33.3	91.1	96.6
UPTURN@16	$L_{\max}$	59.0	83.3	48.3	70.0	53.8	76.7	88.4	97.5	33.9	45.5	92.1	96.4
$\Delta$ (vs. best)		+7.5	+6.7	+12.3	+6.7	+11.2	+10.0	+4.5	+5.0	+8.3	+12.1	+1.0	-0.2
Reflexion@16	$2L_{\max}$	64.2	86.7	49.0	76.7	55.6	80.0	90.2	100.0	34.3	57.6	93.4	97.8

Appendix

Table 14: Full comparison with self-correction baselines , per reasoner and benchmark, at

L_{\max}

and

2L_{\max}

correction budgets.

\Delta

rows give UPTURN’s gain over the best baseline at the same budget; bold marks the best entry per column and budget. SCoRe conditions on the full first attempt, so it is evaluated at

2L_{\max}

only.

Method	Budget	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
		Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
AlphaOne@16	$L_{\max}$	33.8	53.3	21.9	36.7	27.3	46.7	67.0	87.5	16.9	24.2	84.9	92.4
CyclicReflex@16	$L_{\max}$	37.7	56.7	25.2	50.0	28.8	46.7	70.2	92.5	19.1	27.3	86.1	92.6
UPTURN@16	$L_{\max}$	59.0	83.3	48.3	70.0	53.8	76.7	88.4	97.5	33.9	45.5	92.1	96.4
$\Delta$ (vs. best)		+21.2	+26.7	+23.1	+20.0	+25.0	+30.0	+18.3	+5.0	+14.8	+18.2	+6.1	+3.8

Appendix

Table 15: Full comparison with reflection-token injection , per reasoner and benchmark, at

L_{\max}

and

2L_{\max}

correction budgets.

\Delta

rows give UPTURN’s gain over the best injection method at the same budget; bold marks the best entry per column and budget.

Method	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
Reflexion@16 ( $L_{\max}$ )	51.5	76.7	36.0	63.3	42.5	66.7	83.9	92.5	25.6	33.3	91.1	96.6
+ UPTURN	49.0	73.3	35.8	53.3	42.5	60.0	84.4	92.5	23.7	33.3	90.9	97.0
$\Delta$	-2.5	-3.3	-0.2	-10.0	+0.0	-6.7	+0.5	+0.0	-1.9	+0.0	-0.3	+0.4
Reflexion@16 ( $2L_{\max}$ )	64.2	86.7	49.0	76.7	55.6	80.0	90.2	100.0	34.3	57.6	93.4	97.8

Appendix

Table 16: UPTURN on top of a self-correction baseline , per reasoner and benchmark. Each baseline produces its own 16 corrected solutions; UPTURN’s gate then flags the weak ones among them and redirects each once, at the budget named on the baseline row. Reflexion is shown at both

L_{\max}

and

2L_{\max}

; SCoRe conditions on the full first attempt, so only

2L_{\max}

is meaningful for it.

\Delta

rows give the gain of adding UPTURN; bold marks the better entry of the pair.

Method	AIME24		AIME25		AIME26
	Avg	Pass	Avg	Pass	Avg	Pass
Qwen3-4B
Base@32	33.4 $\pm$ 0.8	62.2 $\pm$ 1.9	21.6 $\pm$ 0.2	47.8 $\pm$ 1.9	26.1 $\pm$ 1.2	43.3 $\pm$ 0.0
UPTURN@16	52.1 $\pm$ 1.1	81.1 $\pm$ 3.8	40.0 $\pm$ 1.0	58.9 $\pm$ 1.9	45.8 $\pm$ 1.5	70.0 $\pm$ 0.0
$\Delta$	+18.7	+18.9	+18.4	+11.1	+19.7	+26.7
Qwen3-8B

Appendix

Table 17: Seed robustness. Table 1 repeated with three seeds for training the position policy and drawing rollouts; mean

\pm

std over the three runs.

\Delta

is UPTURN’s gain; bold marks the higher mean.

Method	AIME24		AIME25		AIME26		AMC23		HMMT		MATH500
	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass	Avg	Pass
AceReason-Nemotron-1.1-7B
Base@32	37.2	73.3	27.7	40.0	31.5	56.7	79.7	95.0	19.5	30.3	88.4	96.2
UPTURN@16	47.5	73.3	32.5	46.7	44.6	63.3	89.7	95.0	27.1	45.5	92.1	97.0
$\Delta$	+10.3	0.0	+4.8	+6.7	+13.1	+6.7	+10.0	0.0	+7.6	+15.2	+3.7	+0.8
Skywork-OR1-7B

Appendix

Table 18: Results on two additional reasoners , with the same setup as Table 1 .

\Delta

denotes UPTURN’s gain over Base; bold marks improvements.

LLM Self-Correction

Large language model self-correction studies how language models detect and repair errors in their own answers, reasoning, and actions.

See all 159 papers

The Self-Correction Illusion: LLMs Correct Others but Not Themselves
Jun 4, 2026Kuan-Yen Chen, Fang-Yi Su, Jung-Hsien ChiangLLM Self-Correction LLM Prompting
National Cheng Kung University
In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration
Oct 1, 2025Youngbin Choi, Minjong Lee, Saemi Moon +4LLM Reliability Multi-Turn Language Model Reasoning
Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science and Engineering, POSTECH
What Am I Missing? Question-Answering as Hidden State Probing
May 29, 2026Chu Fei Luo, Samuel Dahan, Xiaodan ZhuLLM Self-Refinement Language Model Self-Assessment
Department of Electrical and Computer Engineering & Ingenuity Labs, Queen’s University · Conflict Analytics Lab, Queen’s University · Cornell Law School +1

Jun 4, 2026J/K move · Enter open · S save

The Self-Correction Illusion: LLMs Correct Others but Not Themselves

Kuan-Yen Chen, Fang-Yi Su, Jung-Hsien Chiang

National Cheng Kung University

LLM Self-Correction LLM Prompting Causal Interventions in Language Models LLM Reasoning

Open paper AskPDF arXiv

Recent work shows that LLM agents struggle to correct errors in their own reasoning traces yet show markedly higher correction rates when identical claims appear under external sources. We ask whether this asymmetry reflects a capability deficit or a role-label artifact: does an agent's willingness to correct a wrong claim depend causally on the chat-template role that carries it, rather than on the claim's content? Our setup keeps the erroneous claim byte-identical across all conditions (SHA-256 verified) and varies only its wrapping role: the agent's own \role{<thought>}, a \role{user} message, a \role{tool} response, or a \role{system <memory>} block. Across 13 model-domain cells covering seven model families and three domains (

n{=}30

paired tasks per cell), relabeling the claim from \role{<thought>} to an external role lifts the explicit-correction rate by 23 to 93 percentage points, with 10 of 13 cells reaching

p{<}0.001

. Further experiments confirm that the effect is asymmetric, mechanistically decomposable, and robust across domains. The failure to self-correct is not a cognitive deficit; it is a chat-template artifact. We exploit this artifact by designing a prompt-structure-only intervention that requires no training and no model modification, with its strongest role label being domain-dependent: \role{<memory>} dominates on math, while a plain \role{user} message dominates on logical deduction.

Abstract

LLM Self-Correction

Explore similar work

The Self-Correction Illusion: LLMs Correct Others but Not Themselves

In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration

What Am I Missing? Question-Answering as Hidden State Probing