Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models

Jean de Dieu Nyandwi; Leena Mathur; Yonatan Bisk; Robert Hawkins; Graham Neubig

cs.CLAug 13, 2026

Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models

Authors: Jean de Dieu Nyandwi, Leena Mathur, Yonatan Bisk, Robert Hawkins, Graham Neubig

Organizations: Carnegie Mellon University · Stanford University

Abstract

Which reasoning behaviors are associated with correct answers in reasoning models, and does reasoning-oriented training amplify those behaviors? This distinction is important because reasoning-oriented training can make traces look more deliberative without amplifying the behaviors most tied to model correctness. We quantify this mismatch with Behavioral Lift, a metric that measures how much correctness changes when a behavior is present versus absent in a model's reasoning trace. Across 15 models and 6 benchmarks spanning text-only and vision-language reasoning, we annotate 15,282 traces with a taxonomy whose core behaviors are defined for both LLM and VLM traces. We find evidence for an Amplification-Lift Gap, in which thinking models strongly amplify self-correction, hypothesis testing, and uncertainty acknowledgment, while the highest-lift behaviors are confidence calibration, knowledge alignment, and self-awareness. Confidence calibration is among the strongest positive signals of correctness in both modalities, yet is barely amplified; uncertainty acknowledgment is amplified by 3--7 $\times$ , yet is weakly or negatively associated with correctness. We find that reasoning-oriented training does not preferentially amplify the highest-Lift behaviors, motivating process-level objectives that reward calibrated and grounded reasoning rather than surface form alone.

Figures & tables

Figure 1: The disconnect between what thinking-oriented training amplifies and what predicts success. Each point is one of the nine cross-modal higher-order behaviors, averaged across VLMs and LLMs ( $N{=}15{,}282$ ). The top-right quadrant is empty: the behaviors most amplified by thinking training are not the ones most associated with correctness.

Behavior	Short definition
Planning	Breaks the problem into explicit sub-steps.
Goal tracking	Tracks progress toward intermediate or final goals.
Hypothesis testing	Considers alternatives, cases, or interpretations.
Self-correction	Revises or corrects a previous step or claim.
Uncertainty ack.	Expresses doubt, ambiguity, or confusion.
Confidence calibration	Certainty tracks the strength of reasoning.

Table 1: Nine behaviors used for cross-modal analysis. Full definitions appear in Appendix Tables 24 and 26 .

Figure 2: Aggregate prevalence of a subset of the nine core higher-order behaviors across all benchmarks per modality. Self-correction, hypothesis testing, and uncertainty acknowledgment show the largest prevalence gaps between thinking and instruct models, while the remaining behaviors are more similar across training types.

Behavior	LLM-Think	LLM-Inst	VLM-Think	VLM-Inst
Confidence calibration	+0.305 ∗	+0.518 ∗	+0.186 ∗	+0.342 ∗
Self-correction	+0.267 ∗	+0.078 ∗	+0.153 ∗	+0.062 ∗
Hypothesis testing	+0.048 ∗	+0.028	-0.005	+0.045
Uncertainty acknowledgment	+0.006	-0.094 ∗	-0.017	-0.004

Table 2: Within-question analysis across language-only and vision-language models. Values are mean per-question

\Delta

accuracy between traces where a behavior is present versus absent, computed from 8 samples per question. Confidence calibration remains among the strongest positive signals, while uncertainty acknowledgment is null or negative. Asterisks indicate bootstrap 95% confidence intervals excluding zero; full results appear in Appendix C , Table 17 (LLMs), and Table 18 (VLMs).

Figure 3: Behavioral Lift for the nine cross-modal higher-order behaviors, ordered to highlight the relationship between amplified behaviors and high-Lift behaviors. Bars show Lift only: confidence calibration ranks highest for VLMs and among the highest for LLMs, while the three behaviors identified as amplified in the prevalence analysis cluster near the bottom. Uncertainty acknowledgment has negative Lift in both modalities.

Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Temp.	Top- $p$	Top- $k$	Min- $p$	Max gen toks
Qwen3	0.6	0.95	20	0	16,384
Qwen2.5-Instruct	0.7	0.8	20	0	16,384
OLMo-3-7B-Think	0.6	0.95	20	0	32,768
OLMo-3-7B-Instruct	0.6	0.95	20	0	32,768
Nemotron-v2	0.6	0.95	20	0	8,192
Nemotron-v2-Base	0.0	–	–	–	8,192

Appendix

Table 3: Generation settings used for the main evaluations. We evaluate LLMs with lm-evaluation-harness and VLMs with lmms-eval , using benchmark-native evaluation wrappers and zero-shot prompting. When a CoT zero-shot benchmark variant is available, we use that variant. We otherwise retain the model-specific defaults and benchmark wrappers provided by the evaluation frameworks, and we do not add extra prompting intended to elicit particular behaviors. Exact run scripts, subset files, prompt templates, and scoring code will be released with the codebase.

		Thinking			Instruct
Behavior	Lift	All	Correct	Incorrect	All	Correct	Incorrect
Confidence calibration	1	.775	.771	–	.849	.889	–
Self-awareness	2	.672	.628	–	.609	.599	–
Knowledge alignment	3	.798	.795	.704	.807	.886	.624
Evidence citation	4	.798	.802	.651	.821	.832	.652
Goal tracking	5	.793	.744	.690	.800	.774	.645

Appendix

Table 4: Linear probe AUC for higher-order behaviors in Qwen3-4B-Thinking and Qwen3-4B-Instruct hidden states. Probes use layer 35 representations from the generated portion of teacher-forced replays, with the best AUC over five trace positions (25%, 50%, 75%, last token, mean pool) reported for each behavior. Dashes indicate insufficient minority-class samples (

<30

). Behaviors are sorted by Behavioral Lift rank from Figure 3 . † marks behaviors amplified at least

3\times

by thinking training.

Accuracy & Recovery (%)
	MATH-500			MMLU-Pro Math			GPQA
	Hi	Base	Lo	Hi	Base	Lo	Hi	Base	Lo
Accuracy	90.6	84.8	80.2	62.0	58.4	46.3	49.8	49.1	45.6
$\Delta$	+5.8 ∗∗∗	–	$-$ 4.6 ∗	+3.6 ∗	–	$-$ 12.1 ∗∗∗	+0.7	–	$-$ 3.5
95% CI	[+2.6,+9.0]		[ $-$ 9.0, $-$ 0.2]	[+0.5,+6.7]		[ $-$ 16.4, $-$ 7.7]	[ $-$ 3.8,+5.1]		[ $-$ 8.3,+0.9]
Recovery	68.9	57.1	46.2	20.2	18.5	13.4	34.8	36.1	28.2

Appendix

Table 5: Prompting results on Qwen3-4B-Thinking across three benchmarks. High-Lift encourages confidence calibration, knowledge alignment, and self-awareness. Low-Lift encourages pervasive uncertainty acknowledgment. Deltas are relative to baseline; significance from paired bootstrap (10,000 resamples, matched by question).

Figure 4: Behavioral Lift (%) within response complexity bins (C2–C4), pooled across all models and benchmarks per modality. The ranking from Figure 3 holds within each bin: confidence calibration remains among the strongest positive associations with correctness and uncertainty acknowledgment shows negative Lift in every bin in both modalities, confirming that the disconnect between amplification and Behavioral Lift is not an artifact of response length or problem difficulty. Word-count quintile analysis in Figure 5 yields consistent results.

Figure 5: Behavioral Lift (%) within word-count quintiles, pooled across all models and benchmarks per modality (VLM: $N{=}7{,}000$ ; LLM: $N{=}8{,}282$ ). Quintile thresholds are computed separately per modality. Dashes indicate insufficient samples ( $N{<}20$ in one group). The top-level pattern from Figure 3 holds: confidence calibration and knowledge alignment dominate, while the behaviors identified as amplified in the prevalence analysis (†) cluster lower. This complements Figure 4 using a purely mechanical binning strategy with no annotator involvement. Appendix D shows that the main prevalence-amplification pattern persists within shared word-count bins, so it is not explained by response length alone.

Figure 6: Recovery Rate across the task spectrum. Thinking models recover from detected failures at 2.3–2.7

\times

the rate of instruct models on benchmarks that reward step-by-step computation (VisualPuzzles, MATH-500, MMLU-Pro). On LogiQA2, instruct models recover better. Recovery advantage tracks task type, not modality.

Figure 7: Recovery rate conditional on behavior presence among traces with at least one detected failure (

N{=}4{,}778

). Confidence calibration, knowledge alignment, and self-correction are the strongest positive predictors of recovery. This links the Behavioral Lift and Recovery Rate analyses: self-correction has modest overall Lift but strongly predicts recovery from failures.

Figure 8: Behavioral Lift (%) by benchmark for VLM and LLM tasks. Each cell computes Lift within a single benchmark. The broad ranking is stable: confidence calibration and knowledge alignment show the strongest positive associations with correctness, while the behaviors identified as amplified in the prevalence analysis cluster near the bottom.

Figure 9: Behavioral Lift (%) by model for all LLMs. Each column computes Lift from a single model’s responses. The Behavioral Lift ordering holds across models: confidence calibration and knowledge alignment typically show the strongest positive associations with correctness, while the behaviors identified as amplified in the prevalence analysis tend to cluster lower.

Figure 10: Behavioral Lift (%) by model for all VLMs. Each column computes Lift from a single model’s responses. The Behavioral Lift ordering holds across models: confidence calibration and knowledge alignment typically show the strongest positive associations with correctness, while the behaviors identified as amplified in the prevalence analysis tend to cluster lower.

Figure 11: Reasoning type composition per benchmark. Each benchmark emphasizes different reasoning demands: MATH-500 is dominated by mathematical and procedural reasoning, LogiQA2 by logical and conceptual reasoning. This variation underlies the task-dependent recovery mechanisms observed across benchmarks.

Figure 12: Conditional co-occurrence of cross-modal higher-order behaviors: $P(\text{column behavior present}\mid\text{row behavior present})$ . Behaviors in the top-left dashed block co-occur at high rates (60–100%), while the three behaviors most amplified by thinking training in the bottom-right dashed block also co-occur (58–80%) but show lower cross-block co-occurrence with the others (15–30%). This suggests the amplified behaviors form a relatively distinct cluster.

Figure 13: Recovery Rate by benchmark for LLM models. Thinking models recover at 2–3

\times

the rate of instruct models on MATH-500 and MMLU-Pro, but instruct models recover better on LogiQA2. The pattern tracks task structure: extended reasoning tasks favor thinking models, while pattern-matching tasks do not.

Figure 14: Behavioral prevalence across model scales for Qwen3-VL on MathVista. The gap in amplified behaviors (left) persists from 2B to 32B parameters. Largely unchanged behaviors (right) converge as model size increases, with instruct models matching or exceeding thinking models at 32B.

Figure 15: Behavioral prevalence across model scales for Qwen3 on MATH-500. The gap in the three behaviors most amplified by thinking training persists from 0.6B to 32B parameters, while the largely unchanged behaviors converge as model size increases, with instruct models matching or exceeding thinking models at 32B.

Model	Variant role	Family	Params	Checkpoint
Vision-Language Models
Qwen3-VL-8B-Thinking	Think-oriented	Qwen	8B	Qwen/Qwen3-VL-8B-Thinking
Qwen3-VL-8B-Instruct	Non-thinking	Qwen	8B	Qwen/Qwen3-VL-8B-Instruct
Kimi-VL-A3B-Thinking	Think-oriented	Kimi	3B ∗	moonshotai/Kimi-VL-A3B-Thinking-2506
Kimi-VL-A3B-Instruct	Non-thinking	Kimi	3B ∗	moonshotai/Kimi-VL-A3B-Instruct
InternVL3.5-8B	Think-oriented	InternVL	8B	OpenGVLab/InternVL3_5-8B

Appendix

Table 6: Models evaluated. The checkpoint column lists the exact public model identifiers used for inference; the variant-role column is descriptive and does not assert that any pair differs by a single isolated training stage. Five released same-family, same-scale thinking/non-thinking endpoint pairs are Qwen3-VL, Kimi-VL, InternVL3.5, Qwen3-4B, and OLMo-3-7B. GLM-4.1V and DeepSeek-R1-Distill-Qwen-7B are unmatched think-oriented auxiliary models; Qwen2.5-7B-Instruct is an additional comparison baseline. Nemotron-Nano-9B-v2 and Nemotron-Nano-9B-v2-Base are included as a base-to-reasoning comparison rather than a clean instruct/thinking endpoint pair. Analyses restricted to same-family thinking/non-thinking endpoint pairs are consistent with the overall results. ∗ Kimi-VL-A3B has 3B active parameters (MoE architecture).

Amplified behaviors (%)
	VLM Benchmarks						LLM Benchmarks
	VisPuzz		MathVista		MMMU		LogiQA2		MATH-500		MMLU-Pro
	Th	In	Th	In	Th	In	Th	In	Th	In	Th	In
Self-correction	50.6	14.9	55.2	4.3	48.7	5.2	21.5	3.3	54.6	12.5	44.5	7.1
Hypothesis testing	42.6	17.6	21.6	3.7	51.5	11.3	25.6	6.4	42.8	12.8	34.4	8.2
Uncertainty ack.	85.1	27.5	51.6	7.2	68.9	10.6	25.0	3.9	43.5	8.1	52.8	10.2

Appendix

Table 7: Behavioral profile of thinking (Th) and instruct (In) models across six benchmarks. Amplified behaviors (top) show large gaps between thinking and instruct models; non-amplified behaviors (middle) show small or reversed gaps. Accuracy and recovery rate (bottom) show where behavior prevalence translates into task performance.

	VLM ( $N{=}7{,}000$ )		LLM ( $N{=}8{,}282$ )
Behavior	Lift	Pres.%	Lift	Pres.%
Conf. calibration	$+$ 72.2	36.9	$+$ 79.6	52.7
Self-awareness	$+$ 62.0	35.9	$+$ 52.7	36.9
Knowledge alignment	$+$ 53.7	56.2	$+$ 80.3	64.0
Evidence citation	$+$ 34.9	54.0	$+$ 49.1	81.7
Goal tracking	$+$ 30.6	50.0	$+$ 52.5	71.7

Appendix

Table 8: Behavioral Lift for the nine cross-modal higher-order behaviors. Rows are grouped to show the amplified behaviors relative to the highest-Lift behaviors. Lift

=P(\text{correct}\mid b{=}\text{true})-P(\text{correct}\mid b{=}\text{false})

. Arrows ( † ) mark behaviors identified as amplified in the prevalence analysis (

{\geq}3\times

thinking/instruct prevalence). These behaviors cluster near the bottom of the Lift ranking.

	All			Thinking			Instruct
Behavior	$P(\checkmark)$	$P(\times)$	$\Delta$	$P(\checkmark)$	$P(\times)$	$\Delta$	$P(\checkmark)$	$P(\times)$	$\Delta$
confidence_cali.	84.7%	0.6%	+84.1	82.0%	1.0%	+81.0	87.5%	0.2%	+87.3
knowledge_alig.	93.9%	15.4%	+78.5	94.4%	18.5%	+75.8	93.4%	12.3%	+81.1
self_awareness	56.7%	4.6%	+52.1	62.2%	6.5%	+55.7	51.0%	2.8%	+48.2
goal_tracking	88.9%	43.7%	+45.2	91.0%	45.1%	+45.9	86.7%	42.3%	+44.4
evidence_cita.	93.5%	62.3%	+31.2	94.4%	60.4%	+34.0	92.6%	64.3%	+28.3

Appendix

Table 9: Conditional prevalence of cross-modal higher-order behaviors in correct versus incorrect LLM traces. Values report

P(b\mid\checkmark)

P(b\mid\times)

, and

\Delta=P(b\mid\checkmark)-P(b\mid\times)

. Confidence calibration and knowledge alignment are strongly enriched in correct traces, while uncertainty acknowledgment is more common in incorrect traces.

Behavior	Post-hoc	No post-hoc	Lucky	Sound
confidence_calibration	3.4%	76.4%	7.6%	94.7%
knowledge_alignment	29.6%	87.2%	41.9%	95.5%
self_awareness	9.1%	66.4%	14.4%	81.5%
goal_tracking	26.1%	76.7%	19.0%	81.5%
evidence_citation	28.7%	82.8%	24.1%	88.2%
planning_present	45.3%	73.5%	27.1%	75.3%

Appendix

Table 10: Behavior prevalence on post-hoc rationalization traces, non-post-hoc traces, lucky guesses, and sound reasoning traces for VLM models. Confidence calibration is almost absent when reasoning is reverse-engineered or lucky, while uncertainty acknowledgment is more common on post-hoc traces than on sound reasoning traces.

Behavior	Post-hoc	No post-hoc	Lucky	Sound
confidence_calibration	1.6%	72.5%	3.4%	95.9%
knowledge_alignment	20.4%	80.9%	42.1%	99.9%
self_awareness	7.3%	48.3%	8.3%	62.8%
goal_tracking	34.7%	86.0%	30.2%	95.9%
evidence_citation	50.5%	93.8%	40.1%	99.8%
planning_present	62.0%	87.2%	44.9%	91.7%

Appendix

Table 11: Behavior prevalence on post-hoc rationalization traces, non-post-hoc traces, lucky guesses, and sound reasoning traces for LLM models. Confidence calibration is almost absent when reasoning is reverse-engineered or lucky, while uncertainty acknowledgment, hypothesis testing, and self-correction are more common on post-hoc traces than on sound reasoning traces.

				Accuracy given $b$		Sample count
Behavior	Lift	Th-Lift	In-Lift	$P(\checkmark\|b)$	$P(\checkmark\|\neg b)$	$N_{b}$	$N_{\neg b}$
Knowledge alignment	$+$ 80.3	$+$ 79.1	$+$ 81.6	90.8	10.5	5299	2983
Confidence calibration	$+$ 79.6	$+$ 76.2	$+$ 83.2	99.6	20.0	4362	3920
Context understanding	$+$ 76.8	$+$ 75.1	$+$ 78.6	87.3	10.5	5542	2740
Logical steps valid	$+$ 69.9	$+$ 68.7	$+$ 71.3	88.8	18.9	5094	3188
Self-awareness	$+$ 52.7	$+$ 54.0	$+$ 52.1	95.2	42.5	3052	5230

Appendix

Table 12: Detailed Behavioral Lift statistics for LLM benchmarks (

N{=}8{,}282

). Lift

=P(\checkmark|b)-P(\checkmark|\neg b)

. Think-Lift and Inst-Lift show lift computed separately on thinking and instruct model pools.

				Accuracy given $b$		Sample count
Behavior	Lift	Th-Lift	In-Lift	$P(\checkmark\|b)$	$P(\checkmark\|\neg b)$	$N_{b}$	$N_{\neg b}$
Confidence calibration	$+$ 72.2	$+$ 69.9	$+$ 74.8	98.8	26.7	2584	4416
Logical steps valid	$+$ 66.5	$+$ 70.5	$+$ 60.6	90.5	23.9	3089	3911
Self-awareness	$+$ 62.0	$+$ 63.6	$+$ 60.5	93.1	31.1	2511	4489
Visual claims accurate	$+$ 60.9	$+$ 63.4	$+$ 56.8	87.8	26.9	3032	3968
Knowledge alignment	$+$ 53.7	$+$ 61.7	$+$ 43.0	76.8	23.1	3933	3067

Appendix

Table 13: Detailed Behavioral Lift statistics for VLM benchmarks (

N{=}7{,}000

). Lift

=P(\checkmark|b)-P(\checkmark|\neg b)

. Think-Lift and Inst-Lift show lift computed separately on thinking and instruct model pools.

Behavior	Lift	$P(\checkmark\mid b)$	$P(\checkmark\mid\neg b)$	Present %	$N_{p}$	$N_{a}$
Knowledge alignment	+84.2	94.6	10.4	66.7	925	461
Confidence calibration	+77.9	99.9	22.0	57.3	794	592
Goal tracking	+67.7	79.2	11.6	81.3	1127	259
Self-awareness	+58.6	96.9	38.3	48.3	670	716
Evidence citation	+51.4	71.3	19.8	90.9	1260	126
Planning present	+20.2	67.1	46.9	97.7	1354	32

Appendix

Table 14: Behavioral Lift for seven frontier models on GPQA-Diamond, computed from visible responses only and pooled across models. The same broad ranking appears as in the main analysis: knowledge alignment and confidence calibration are the strongest positive signals, hypothesis testing remains near zero, and uncertainty acknowledgment remains negative.

Behavior	Example surface markers	Qualitative pattern
Planning present	“let me break this down,” “first…then…,” “let me define…,” “step 1 / step 2,” explicit headers	Often marked by explicit upfront decomposition of the task into steps, variables, or constraints before the main reasoning begins.
Hypothesis testing	“alternatively,” “what if,” “suppose,” “case 1 / case 2,” “if X…if not X…”	Often marked by explicit branching over alternative interpretations or scenarios, rather than generic linear continuation.
Self-correction	“wait,” “actually,” “that’s not correct,” “let me rethink,” “let’s start over”	Often marked by explicit revision and backtracking after a contradiction, detected error, or failed line of reasoning.
Uncertainty acknowledgment	“I’m not sure,” “I’m confused,” “maybe,” “am I missing something?”	Often marked by explicit hedging and self-doubt language, including admissions of ambiguity or difficulty.
Evidence citation	“the passage states,” “given that,” “from condition 1,” “based on the fact that…”	Often marked by explicit grounding of a claim in a specific fact, premise, or constraint from the prompt.
Confidence calibration	“I’m not sure,” “maybe,” “let me double-check,” “must be,” “therefore, in all cases”	Often marked not by isolated hedging (which signals uncertainty acknowledgment), but by a shift in expressed certainty that tracks the strength of the reasoning — increasing confidence when steps succeed, decreasing when they falter.

Appendix

Table 15: Qualitative surface markers of annotated higher-order behaviors. For each behavior, we manually reviewed behavior-positive traces and summarized the most common recurring surface markers and broader discourse pattern. These summaries are qualitative and intended to illustrate how the behaviors tend to appear in visible reasoning traces.

Cross-modal failure modes (%)
	VLM Benchmarks						LLM Benchmarks
	VisPuzz		MathVista		MMMU		LogiQA2		MATH-500		MMLU-Pro
	Th	In	Th	In	Th	In	Th	In	Th	In	Th	In
Logical failure	77.6	72.7	31.9	55.3	44.6	53.1	39.7	44.4	28.6	29.1	51.1	41.4
Shortcut	52.7	43.1	23.2	49.3	27.7	45.9	20.3	34.2	16.4	19.0	18.2	18.3
Post-hoc rational.	79.1	61.2	35.1	48.7	47.9	51.7	33.1	34.8	22.9	14.2	40.0	23.2

Appendix

Table 16: Failure mode rates, recovery rate, and accuracy across six benchmarks. cross-modal failures (top) are directly comparable across modalities. VLM-specific and LLM-specific failures (middle) apply only to their respective benchmarks; gray cells indicate non-applicable modality. Recovery Rate

=P(\text{correct}\mid\text{any failure detected})

. On reasoning-heavy benchmarks, thinking models recover at 2.3–2.7

\times

the rate of instruct models. On LogiQA2, instruct models recover better.

Behavior	Mean $\Delta$	95% CI	$N$	Mean $\Delta$	95% CI	$N$
	Qwen3-4B-Think			Qwen3-4B-Instruct
Confidence calibration	+0.305	[+0.254, +0.363]	189	+0.518	[+0.453, +0.582]	145
Self-awareness	+0.187	[+0.150, +0.226]	225	+0.158	[+0.131, +0.189]	227
Knowledge alignment	+0.596	[+0.524, +0.673]	110	+0.728	[+0.651, +0.794]	91
Evidence citation	+0.299	[+0.201, +0.388]	78	+0.268	[+0.143, +0.393]	54
Goal tracking	+0.405	[+0.331, +0.481]	112	+0.434	[+0.346, +0.516]	99

Appendix

Table 17: Within-question paired analysis on MATH-500. For each of 250 questions, we generate 8 traces at temperature 0.6 and compute the per-question accuracy difference between traces where a behavior is present versus absent. Reported values are mean within-question

\Delta

accuracy, with bootstrap 95% confidence intervals and the number of questions for which both present and absent traces were observed.

Behavior	Mean $\Delta$	95% CI	$N$	Mean $\Delta$	95% CI	$N$
	Qwen3-4B-VL-Think			Qwen3-4B-VL-Instruct
Confidence calibration	+0.186	[+0.137, +0.240]	115	+0.342	[+0.235, +0.452]	62
Self-awareness	+0.136	[+0.094, +0.180]	178	+0.131	[+0.082, +0.190]	130
Knowledge alignment	+0.166	[+0.112, +0.216]	134	+0.109	[+0.054, +0.164]	96
Evidence citation	+0.149	[+0.103, +0.200]	149	+0.107	[+0.051, +0.168]	95
Goal tracking	+0.080	[+0.042, +0.121]	172	+0.057	[+0.020, +0.101]	108

Appendix

Table 18: Within-question paired analysis on MathVista for Qwen3-4B-VL-Think and Qwen3-4B-VL-Instruct. For each question, we generate 8 traces and compare mean accuracy between traces where a behavior is present versus absent on the same question. Reported values are mean within-question

\Delta

accuracy, with bootstrap 95% confidence intervals and the number of questions for which both present and absent traces were observed.

	2B		4B		8B		32B
	Th	In	Th	In	Th	In	Th	In
Self-correction	64.0	4.7	72.7	10.3	69.3	11.3	61.3	11.3
Hypothesis testing	17.3	2.0	31.0	8.7	28.0	9.7	17.7	11.0
Uncertainty ack.	66.7	8.3	77.7	13.0	61.7	14.3	43.7	15.3
Self-awareness	45.7	25.3	50.3	46.0	64.7	39.7	66.0	63.3
Conf. calibration	40.7	32.0	43.3	56.3	61.0	51.3	71.7	78.0

Appendix

Table 19: Scaling analysis: Qwen3-VL (thinking) vs. Qwen3-VL (instruct) on MathVista, 2B to 32B parameters. Amplified behaviors show persistent gaps at all scales. Largely unchanged behaviors converge, with instruct matching or exceeding thinking at 32B.

	0.6B		1.7B		4B		8B		14B		32B
	Th	In	Th	In	Th	In	Th	In	Th	In	Th	In
Self-correction	63.4	0.0	60.3	1.7	82.6	2.0	81.4	4.6	92.3	1.7	61.4	2.3
Hypothesis testing	46.6	0.6	42.3	1.1	55.7	3.1	64.3	4.6	74.3	3.7	46.3	4.3
Uncertainty ack.	67.1	1.4	53.7	1.1	58.3	0.0	53.4	0.6	59.7	0.3	29.4	0.3
Self-awareness	22.0	8.3	22.0	12.0	45.7	27.1	43.4	28.0	69.1	26.3	54.9	33.1
Conf. calibration	32.3	19.1	39.7	30.9	60.9	58.3	54.9	69.1	76.0	74.0	77.1	78.9

Appendix

Table 20: Scaling analysis: Qwen3 (thinking) vs. Qwen2.5 (instruct) on MATH-500, 0.6B to 32B parameters. The same patterns hold as in the VLM scaling analysis: amplified behavior gaps persist across scale, largely unchanged behaviors converge, self-correction Lift diminishes while confidence calibration Lift remains high.

Table 21: Sensitivity of LLM Behavioral Lift to simulated random annotation noise. The first subtable shows the clean baseline with no injected noise. At each nonzero noise level, we randomly flip a fixed fraction of behavior labels and recompute Lift over 1000 trials. Each noisy subtable reports the clean Lift, the mean noisy Lift, the 95% range across trials, the mean rank, and the fraction of trials that preserve the original sign. Behaviors are kept in the clean baseline order for easier comparison across noise levels.

Table 22: Sensitivity of VLM Behavioral Lift to simulated random annotation noise. The first subtable shows the clean baseline with no injected noise. At each nonzero noise level, we randomly flip a fixed fraction of behavior labels and recompute Lift over 1000 trials. Each noisy subtable reports the clean Lift, the mean noisy Lift, the 95% range across trials, the mean rank, and the fraction of trials that preserve the original sign. Behaviors are kept in the clean baseline order for easier comparison across noise levels.

Behavior	$N$	Mean beh. pos.	Before answer	Mean gap
Hypothesis testing	266	26.7%	97.0%	65.0 pp
Self-correction	300	34.0%	99.0%	60.0 pp
Uncertainty acknowledgment	253	35.2%	95.3%	56.2 pp
Confidence calibration	300	34.8%	91.0%	47.8 pp

Appendix

Table 23: Temporal position of behaviors relative to answer commitment in thinking-model traces. All four behaviors typically appear before the answer, suggesting this pattern reflects the general structure of reasoning traces rather than a property unique to calibration.

Table 24: LLM annotation taxonomy: reasoning quality, metacognitive behaviors, and reasoning types.

Table 25: LLM annotation taxonomy: failure modes and summary metrics.

Table 26: VLM annotation taxonomy: visual grounding, reasoning quality, and advanced/metacognitive behaviors.

Table 27: VLM annotation taxonomy: reasoning types, failure modes, and summary metrics.

Large Reasoning Models

Large reasoning models are language models trained or prompted to solve difficult tasks through extended, structured computation.

See all 339 papers

RL-ARC: Calibrating Large Reasoning Models via Reasoning-guided Uncertainty
Oct 8, 2026Gukhyeon Lee, SangKeun LeeLanguage Model Calibration LLM Uncertainty Estimation
Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea · Department of Computer Science and Engineering, Korea University, Seoul, Republic of Korea
Understanding and Mitigating Premature Confidence for Better LLM Reasoning
May 23, 2026Jingchu Gai, Guanning Zeng, Christina Baek +4Confidence Estimation in Language Models RL for Language Model Reasoning
1Carnegie Mellon University · 2Tsinghua University
Enhancing LLM Metacognition via Cognitive Pairwise Training
May 30, 2026Weitao Li, Hao Zhou, Xuanyu Lei +11Pairwise Preference Learning RL for Language Model Reasoning
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China · Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China · WeChat AI, Tencent Inc., China

Oct 8, 2026J/K move · Enter open · S save

RL-ARC: Calibrating Large Reasoning Models via Reasoning-guided Uncertainty

Gukhyeon Lee, SangKeun Lee

Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea · Department of Computer Science and Engineering, Korea University, Seoul, Republic of Korea

Language Model Calibration LLM Uncertainty Estimation Large Reasoning Models RL for Language Model Reasoning

Open paper AskPDF arXiv

Language models (LMs) are commonly trained with Reinforcement Learning with Verifiable Rewards (RLVR) to enhance their reasoning capabilities. However, since RLVR does not explicitly account for calibration during training, it can lead to severe calibration degradation, including overconfidence. Recent calibration-aware training methods for LMs, which incorporate objectives for uncertainty estimation into training, improve calibration but still exhibit overconfidence under distribution shift, while sacrificing reasoning performance. To this end, we propose RL-ARC, a calibration-aware training framework that jointly leverages reasoning confidence and answer confidence. Specifically, RL-ARC leverages reasoning confidence as an auxiliary signal for calibrating answer confidence, applying it as reasoning-guided regularization for correct cases and as an overconfidence penalty for incorrect cases. Comprehensive results across ID and OOD settings show that, beyond improving calibration, RL-ARC enables reasoning models to adaptively estimate confidence based on the given question without substantially sacrificing reasoning performance, thereby highlighting the importance of reasoning confidence for training reliable reasoning models.

Abstract

Large Reasoning Models

Explore similar work

RL-ARC: Calibrating Large Reasoning Models via Reasoning-guided Uncertainty

Understanding and Mitigating Premature Confidence for Better LLM Reasoning

Enhancing LLM Metacognition via Cognitive Pairwise Training