Decoding the Critique Mechanism in Large Reasoning Models

Hoang Phan; Quang H. Nguyen; Hung T. Q. Le; Xiusi Chen; Heng Ji; Khoa D. Doan

cs.LGMar 17, 2026

Decoding the Critique Mechanism in Large Reasoning Models

Authors: Hoang Phan, Quang H. Nguyen, Hung T. Q. Le, Xiusi Chen, Heng Ji, Khoa D. Doan

Organizations: VinUni-Illinois Smart Health Center, VinUniversity · University of Illinois Urbana-Champaign

Abstract

Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions, yielding strong performance on complex logical benchmarks. We hypothesize that such behaviors are beneficial only when the model has sufficiently strong ``critique'' ability to detect its own mistakes. This work systematically investigates how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps. Notably, we discover a peculiar yet important phenomenon: despite the error propagating throughout the entire chain-of-thought (CoT) without any verbalized correction, the model still reaches the correct final answer after the thinking process finishes. This recovery implies the existence of an internal mechanism helping the model to detect errors and trigger self-correction, which we refer to as the hidden critique ability. Building on feature space analysis, we identify a highly interpretable critique vector representing this behavior. Extensive experiments across multiple model scales and families demonstrate that steering latent representations with this vector improves the model's error detection capability and enhances the performance of test-time scaling at no extra training cost. Our findings provide a valuable understanding of LRMs' critique behavior, suggesting a promising direction to control and improve their self-verification mechanism. Our code is available at: https://github.com/mail-research/lrm-critique-vectors.

Figures & tables

Figure 1: Hidden self-correction despite erroneous reasoning in R1-32B. Left: original correct reasoning. Right: injected error (3 + 4 = 6 ) propagates to incorrect thinking conclusion ( $20 ), yet the model recovers to the correct final answer ( $18 ). We hypothesize this indicates implicit error detection beyond the observable CoT. Full generation details are in Appendix J.1 . — Figure 1: Hidden self-correction despite erroneous reasoning in R1-32B. Left: original correct reasoning. Right: injected error (3 + 4 = 6 ) propagates to incorrect thinking conclusion ( $20 ), yet the model recovers to the correct final answer ($ 18 ). We hypothesize this indicates implicit error detection beyond the observable CoT. Full generation details are in Appendix J.1 .

Figure 2: Distribution of reasoning-result alignment across R1 and Qwen3 model variants. The proportions represent four distinct outcomes based on the correctness of the internal “thinking” process versus the correctness of the final “answer” on the GSM8K-Error and MATH500-Error benchmarks. In this paper, we analyze the self-correction mechanisms represented by the light blue segments: ‘

\times

Think

\checkmark

Answer’.

Cases	GSM8K				MATH500
Cases	Q3-4B	R1-8B	R1-14B	R1-32B	Q3-4B	R1-8B	R1-14B	R1-32B
$\times$ Think, $\times$ Ans	$3.9$ %	$15.6$ %	$6.8$ %	$5.0$ %	$1.5$ %	$5.0$ %	$4.1$ %	$2.4$ %
$\checkmark$ Think, $\checkmark$ Ans	$95.8$ %	$81.8$ %	$92.4$ %	$94.0$ %	$98.3$ %	$95.0$ %	$95.5$ %	$97.4$ %
$\checkmark$ Think, $\times$ Ans	$0.0$ %	$0.7$ %	$0.2$ %	$0.5$ %	$0.2$ %	$0.0$ %	$0.0$ %	$0.0$ %
$\times$ Think, $\checkmark$ Ans	$0.2$ %	$1.9$ %	$0.5$ %	$0.5$ %	$0.0$ %	$0.0$ %	$0.4$ %	$0.2$ %

Table 1: Distribution of reasoning and final answer correctness on GSM8K and MATH500.

Model	GSM8K		MATH500		Layer
	AUROC	ECE	AUROC	ECE
Qwen3-4B	1.000	0.002	1.000	0.026	21
R1-8B	0.998	0.019	0.996	0.022	31
R1-14B	1.000	0.003	0.998	0.012	28
R1-32B	1.000	0.002	0.997	0.008	38

Table 2: Linear probing results. Comparison of AUROC and ECE evaluated on GSM8K (ID) and MATH500 (OOD) across four different models. The reported results are selected for the best layer. Detailed results can be found in Appendix F .

	Token	Logit	Token	Logit
Rank	Qwen3-4B		R1-32B
1	wrongly	1.78	Wait	2.68
2	Nope	1.72	但实际上 (But actually)	2.65
3	而且还 (And also)	1.63	却 (But/Yet)	2.64
4	really	1.61	however	2.60
5	_that	1.60	but	2.58

Table 3: Comparison of the top 10 predicted tokens and their associated logit scores for the Qwen3-4B and R1-32B models.

Panel of Figure 3 — Figure 3: Effect of steering coefficient $\alpha$ on error detection accuracy, correct solution accuracy, and F1 score. The monotonic relationship between $\alpha$ and error detection holds on both ProcessBench and BIG-Bench Mistake. The dashed vertical line indicates the baseline performance at $\alpha=0$ .

Panel of Figure 4 — Figure 4: Qualitative example of positive steering on BIG-Bench arithmetic task with R1-32B. There is a subtle arithmetic error introduced in the prompt ( $7+2+20=30$ , whereas it should be $29$ ). While the baseline model hallucinates that the math is correct during verification, positive steering enables the model to successfully catch the addition error.

Figure 5: Impact of steering on model accuracy for test-time scaling. Left to right: Accuracy across three benchmarks: GSM8K-Error, MATH500-Error, and BIG-Bench Mistake. For increasing iterations of test-time scaling, positive steering consistently enhances the performance of all models.

Model	AIME24			AIME25			GSM8K			MATH500
	$-1$	$0$	$+1$	$-1$	$0$	$+1$	$-1$	$0$	$+1$	$-1$	$0$	$+1$
Qwen3-4B	50.0	56.7	56.7	50.0	46.7	53.3	50.8	57.7	62.9	92.2	93.0	92.8
R1-8B	40.0	50.0	50.0	30.0	30.0	30.0	69.4	70.7	70.7	89.2	89.9	89.9
R1-14B	56.7	56.7	60.0	36.7	46.7	46.7	68.6	71.3	70.3	92.8	94.2	95.6
R1-32B	60.0	63.3	66.7	50.0	53.3	53.3	89.1	89.2	89.6	94.2	94.3	95.0

Table 4: Steering accuracy (%) on standard (unperturbed) reasoning benchmarks, using the same GSM8K-Error-derived vector as Figure 5 . Columns under each dataset correspond to steering coefficient

\alpha\in\{-1,0,+1\}

; bold marks the best value within each row’s dataset block.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Wait-once. The model fails to recover in the thinking trace, but still corrects the final answer.

Panel of Figure 7 — Figure 7: Misprint. Thinking stops immediately after the injected error and proceeds directly to the final answer.

Panel of Figure 8 — Figure 8: Question-masked. The question is masked, leaving only the corrupted reasoning trace as context.

Panel of Figure 9 — Figure 9: Answer-prefix. The model is instructed to revisit the thinking trace in its final response.

Dataset	Model	Intervened
-local	Wait
-once	Misprint	Question
-masked	Answer
-prefix
GSM8K	Qwen3-4B	20.12	0 3.03	80.52	12.13	12.21
	R1-8B	55.12	21.76	67.32	41.39	31.61

Appendix

Table 5: Recovery rate (

\times

Think,

\checkmark

Final, in %) across error-injected variants.

Model	Original GSM8K-test	Synthetic GSM8K-test
Qwen3-4B	20.12	48.72
R1-8B	55.12	68.54
R1-14B	77.43	71.23
R1-32B	74.00	71.58

Appendix

Table 6: Recovery rate (

\times

Think,

\checkmark

Final, in %) of the original and synthetic intervention on GSM8K-test.

	Token	Logit	Token	Logit	Token	Logit	Token	Logit	Token	Logit
Rank	Layer 29		Layer 30		Layer 31		Layer 32		Layer 33
1	wrongly	1.78	而且还 (And also)	2.28	really	2.49	wrongly	2.88	!)\n	3.10
2	Nope	1.72	Nope	2.21	wrongly	2.45	!)\n	2.69	wrongly	2.91
3	而且还 (And also)	1.63	wrongly	2.17	而且还 (And also)	2.42	—but	2.65	!)\n\n	2.80
4	really	1.61	really	2.11	Nope	2.38	而且还 (And also)	2.63	)!\n	2.79
5	_that	1.60	!)\n	2.02	Actually	2.22	rethink	2.62	!!\n\n	2.77

Appendix

Table 7: Evolution of top 10 predicted tokens and logit scores across five layers (Layer 29 to Layer 33) for the Qwen3-4B model.

	Token	Logit	Token	Logit	Token	Logit	Token	Logit
Rank	Layer 55		Layer 56		Layer 57		Layer 58
1	但实际上 (But actually)	2.83	但实际上 (But actually)	2.66	Wait	2.68	Wait	2.95
2	but	2.54	Wait	2.60	但实际上 (But actually)	2.65	Wait	2.78
3	Wait	2.43	Wait	2.46	却 (But/Yet)	2.64	however	2.78
4	但由于 (But due to)	2.42	however	2.46	however	2.60	(	2.74
5	但现在 (But now)	2.38	却 (But/Yet)	2.45	but	2.58	却 (But/Yet)	2.70

Appendix

Table 8: Evolution of top 10 predicted tokens and logit scores across four layers (Layer 55 to Layer 58) for the R1-32B model.

Figure 10: Layer-wise steering results. The separation between the performance of positive and negative steering is minimal for early and late layers. For middle layers, this performance gap is the most noticeable.

Model	$\alpha=0.0$	$\alpha=-1.0$	$\alpha=1.0$	$\Delta$	Layer
Qwen3-4B	42.36	25.00	63.72	38.72	21
R1-8B	57.03	53.36	73.48	20.13	13
R1-14B	80.99	73.71	85.64	11.93	28
R1-32B	79.42	69.74	86.25	16.51	44

Appendix

Table 9: Results of the optimal layer. Overall accuracy (%) of the optimal layer evaluated on the training split of GSM8K for baseline (

\alpha=0.0

), positive steering (

\alpha=1.0

), negative steering (

\alpha=-1.0

), and the difference between positive and negative steering (

\Delta

Figure 11: Layer-wise linear probing results. Left to right: AUROC and ECE for GSM8K (in-distribution) and MATH500 (out-of-distribution). For both settings, the linear prober can achieve near-perfect AUROC and low ECE at several layers.

	Unrecovered	Recovered	Unrecovered	Recovered	Unrecovered	Recovered
Model	Original		Question masking		Misprinting
Qwen3-4B	64.22	99.53	67.58	99.38	65.27	100.00
R1-8B	67.38	99.78	66.00	99.45	69.96	99.61
R1-14B	68.49	98.65	45.97	99.76	68.64	100.00
R1-32B	67.82	99.75	33.71	99.88	67.82	99.80

Appendix

Table 10: Rate (%) of held-out activations classified as recovered by the base probe of Section 4.2 . A random classifier scores around

50\%

		AUROC	ECE	AUROC	ECE
Model	Layer	GSM8K		MATH500
Qwen3-4B	21	0.9911	0.0262	0.9557	0.1154
R1-8B	13	0.9146	0.0676	0.8560	0.0580
R1-14B	28	0.8856	0.0732	0.8776	0.2532
R1-32B	44	0.9122	0.0771	0.8801	0.2524

Appendix

Table 11: Linear separability of recovered vs. unrecovered intervened trajectories.

		$\alpha{=}{-}1$	$\alpha{=}0$	$\alpha{=}{+}1$	$\alpha{=}{-}1$	$\alpha{=}0$	$\alpha{=}{+}1$
Model	Method	Accuracy (%)			F1 (%)
Qwen3-4B	Ours	↓ -4.19	78.69	↑ +2.48	↓ -4.85	80.26	↑ +2.37
	Embed.	↓ -0.34	–	↑ +0.39	↓ -0.38	–	↑ +0.38
R1-8B	Ours	↓ -1.48	40.46	↑ +2.53	↓ -1.94	41.16	↑ +2.08
	Embed.	↓ -0.79	–	↓ -2.18	↓ -0.99	–	↓ -2.54
R1-14B	Ours	↓ -1.57	72.08	↑ +1.75	↓ -1.82	73.14	↑ +2.05

Appendix

Table 12: Accuracy and F1 (%) on ProcessBench.

\alpha{=}0

is the unsteered baseline.

		$\alpha{=}{-}1$	$\alpha{=}0$	$\alpha{=}{+}1$	$\alpha{=}{-}1$	$\alpha{=}0$	$\alpha{=}{+}1$
Model	Method	Accuracy (%)			F1 (%)
Qwen3-4B	Ours	↓ -3.75	54.27	↑ +3.59	↓ -5.44	55.11	↑ +4.60
	Embed.	↑ +0.16	–	↑ +1.90	↑ +0.08	–	↑ +1.57
R1-8B	Ours	↓ -5.13	25.59	↑ +7.32	↓ -6.31	25.82	↑ +7.47
	Embed.	↓ -5.04	–	↓ -1.82	↓ -7.45	–	↓ -2.64
R1-14B	Ours	↓ -1.71	53.41	↑ +1.26	↓ -3.03	50.74	↑ +1.20

Appendix

Table 13: Accuracy and F1 (%) on BIG-Bench Mistake.

\alpha{=}0

is the unsteered baseline.

Model	Source	$-1.0$	$-0.8$	$-0.6$	$-0.4$	$-0.2$	$\alpha{=}0$	$+0.2$	$+0.4$	$+0.6$	$+0.8$	$+1.0$
Qwen3-4B	GSM8K	↓ -5.05	↓ -3.90	↓ -2.75	↓ -2.20	↓ -0.50	73.45	↓ -0.50	↑ +1.70	↑ +1.75	↑ +1.60	↑ +1.80
	MATH500	↓ -4.90	↓ -4.00	↓ -3.10	↓ -1.70	↓ -2.65		↑ +0.35	↑ +1.70	↑ +1.90	↑ +0.95	↑ +2.40
R1-8B	GSM8K	↓ -3.55	↓ -3.45	↓ -1.70	↓ -1.90	↓ -0.75	45.15	↑ +1.80	↑ +0.30	↑ +1.55	↑ +3.10	↑ +4.85
	MATH500	↓ -7.95	↓ -7.75	↓ -4.75	↓ -3.10	↓ -0.65		↑ +1.65	↑ +1.65	↑ +3.20	↑ +2.30	↑ +1.55
R1-14B	GSM8K	↓ -1.60	↓ -1.35	↓ -0.20	↓ -1.45	↓ -0.65	65.15	↑ +1.40	↑ +1.50	↑ +1.05	↑ +2.05	↑ +2.45
	MATH500	↓ -16.15	↓ -12.35	↓ -9.60	↓ -4.70	↓ -2.30		↑ +2.50	↑ +2.60	↑ +3.50	↑ +5.00	↑ +4.85

Appendix

Table 14: Accuracy (%) on ProcessBench under critic vectors sourced from GSM8K vs. MATH-500.

Model	Source	$-1.0$	$-0.8$	$-0.6$	$-0.4$	$-0.2$	$\alpha{=}0$	$+0.2$	$+0.4$	$+0.6$	$+0.8$	$+1.0$
Qwen3-4B	GSM8K	↓ -5.25	↓ -4.00	↓ -2.79	↓ -2.22	↓ -0.47	77.02	↓ -0.72	↑ +1.26	↑ +1.19	↑ +1.00	↑ +1.10
	MATH500	↓ -5.09	↓ -4.12	↓ -3.18	↓ -1.67	↓ -2.72		↑ +0.25	↑ +1.54	↑ +1.42	↑ +0.30	↑ +1.74
R1-8B	GSM8K	↓ -3.00	↓ -3.10	↓ -1.44	↓ -1.70	↓ -1.00	46.64	↑ +1.70	↓ -0.19	↑ +1.00	↑ +2.30	↑ +4.60
	MATH500	↓ -10.17	↓ -8.38	↓ -4.53	↓ -2.58	↓ -0.48		↑ +1.30	↑ +1.30	↑ +1.46	↑ +0.88	↓ -0.43
R1-14B	GSM8K	↓ -1.88	↓ -1.61	↓ -0.30	↓ -1.53	↓ -0.68	68.38	↑ +1.44	↑ +1.60	↑ +1.15	↑ +2.06	↑ +2.56
	MATH500	↓ -21.75	↓ -15.94	↓ -11.89	↓ -5.68	↓ -2.56		↑ +2.61	↑ +2.71	↑ +3.49	↑ +4.98	↑ +4.74

Appendix

Table 15: F1 (%) on ProcessBench under critic vectors sourced from GSM8K vs. MATH-500.

Panel of Figure 12 — Figure 12: Complete example shown in Figure 1 generated by DeepSeek-R1-Distill-Qwen-32B (R1-32B). As laid out in Section 3.1 , we generate the arithmetic error with GPT-5 (highlighted by a box in the right column) and insert this error immediately after the $\langle$ think $\rangle$ token, which causes the model to make subsequent mistakes (highlighted in red ) in the thinking stage. However, for the final answers, the model is able to recover from those thinking mistakes and derive the correct final solution. In this example, correct thinking steps and solutions are highlighted in green .

Panel of Figure 13 — Figure 13: Qualitative example of positive steering on a GSM8K question from ProcessBench with R1-32B model. The proposed solution makes an unjustified assumption about the work schedule (5 days vs 7 days). The Baseline model accepts this assumption as “standard”, while the steered model critically analyzes the specific phrasing “entire week” and identifies the discrepancy.

Panel of Figure 14 — Figure 14: Qualitative example on GSM8K task with Qwen3-4B model. The baseline model hallucinates a logic error, incorrectly arguing that the total budget ( $48) should be used for the counterfactual, ignoring that the CD was still purchased. The positive steering helps the model successfully verify the correct logic: money saved equals the cost of the unpurchased item ($ 44).

Panel of Figure 15 — Figure 15: Qualitative example of negative steering on ProcessBench-GSM8K with R1-32B model. The prompt contains a logical error in the first step (calculating total food prepared). The baseline model successfully debugs the setup ( $54$ hotdogs vs $36$ ), whereas negative steering suppresses this critical check and accepts the erroneous premise.

Panel of Figure 16 — Figure 16: Qualitative example of positive steering on GSM8K-Error with R1-32B model. The baseline model uses a multiplication error ( $5\times 8=30$ ) and confirms it during self-verification ( $70+30=100$ ). Positive steering initially makes the same error but successfully catches it during the “Wait” phase, correcting the allowance to $40$ and the final answer to $60$ .

Panel of Figure 17 — Figure 17: Qualitative example of negative steering on ProcessBench-GSM8K with R1-32B model. The prompt context introduces an arithmetic error ( $3+$ 2= $6 instead of$ 5). The baseline model initially adopts the error but triggers a self-correction after generating “Wait,” re-evaluating the costs. The negatively steered model accepts the erroneous premise and justifies the incorrect result.

Language Model Self-Verification

See all 16 papers

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models
Jun 1, 2026Simone Caldarella, Davide Talon, Rahaf Aljundi +2LLM Evaluation Overthinking in Language Models
University of Trento · 3Fondazione Bruno Kessler · 2Toyota Motor Europe
An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models
May 31, 2026Mingzhong Sun, Teresa Yeo, Armando Solar-Lezama +1Reasoning Evaluation LLM Interpretability
1NUS Department of Computer Science · 4Singapore-MIT Alliance for Research and Technology (SMART) · 2MIT EECS +1
Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models
Sep 29, 2025Qingjie Zhang, Yujia Fu, Yang Wang +5Efficient Language Model Reasoning Efficient Language Model Inference
Tsinghua University · 2Ant Group

Jun 1, 2026J/K move · Enter open · S save

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models

Simone Caldarella, Davide Talon, Rahaf Aljundi, Elisa Ricci+1

University of Trento · 3Fondazione Bruno Kessler · 2Toyota Motor Europe

LLM Evaluation Overthinking in Language Models Reasoning Evaluation LLM Reliability Large Reasoning Models Reasoning Trace Analysis

Open paper AskPDF arXiv

Large Reasoning Models (LRMs) improve performance by generating explicit intermediate reasoning traces through increased test-time compute, yet the assumption that longer reasoning is consistently beneficial remains under-examined. While recent evidence shows that additional reasoning can lead models to overthink, we ask: "Once a model has reached the correct answer, does further reasoning refine the solution, or deviate from it?" To study the dynamics after correctness, we introduce a prefix-level trajectory evaluation protocol grounded in reasoning sufficiency, defining the minimum reasoning budget required for a model to first generate the correct answer. This allows us to disentangle verbose overthinking, where additional reasoning is redundant but harmless, from harmful overthinking, where continued reasoning destabilizes an already-correct trajectory. Starting from multimodal benchmarks, we find that many instances considered reasoning-intensive require surprisingly little reasoning. Moreover, stopping at the first correct prefix improves accuracy over standard reasoning up to 21%, revealing that current models are limited not only by their ability to reason, but also by their inability to stop at the right time. Furthermore, while common efficiency strategies like early stopping substantially reduce verbose overthinking (up to 50%), they fail to mitigate harmful overthinking. Failure analysis reveals that correctness deviations are mainly driven by logical drift and visual reinterpretation. Finally, we show that our findings generalize to language-only reasoning benchmarks, highlighting harmful overthinking as a broader reliability risk. Code available at https://simonecaldarella.github.io/thinking-past-the-answer.

Abstract

Language Model Self-Verification

Explore similar work

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models

An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models

Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models