When Shallow Wins: Silent Failures and the Depth-Accuracy Paradox in Latent Reasoning

Subramanyam Sahoo; Aman Chadha; Vinija Jain; Divya Chaudhary

cs.LGMar 3, 2026

When Shallow Wins: Silent Failures and the Depth-Accuracy Paradox in Latent Reasoning

Authors: Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary

Organizations: Independent · AWS Generative AI Innovation Center, Amazon Web Services · Stanford University · Meta AI ·

Abstract

Mathematical reasoning models are widely deployed in education, automated tutoring, and decision support systems despite exhibiting fundamental computational instabilities. We demonstrate that state-of-the-art models (Qwen2.5-Math-7B) achieve 61% accuracy through a mixture of reliable and unreliable reasoning pathways: 18.4% of correct predictions employ stable, faithful reasoning while 81.6% emerge through computationally inconsistent pathways. Additionally, 8.8% of all predictions are silent failures -- confident yet incorrect outputs. Through comprehensive analysis using novel faithfulness metrics, we reveal: (1) reasoning quality shows weak negative correlation with correctness (r=-0.21, p=0.002), reflecting a binary classification threshold artifact rather than a monotonic inverse relationship; (2) scaling from 1.5B to 7B parameters (4.7x increase) provides zero accuracy benefit on our evaluated subset (6% of GSM8K), requiring validation on the complete benchmark; and (3) latent reasoning employs diverse computational strategies, with ~20% sharing CoT-like patterns. These findings highlight that benchmark accuracy can mask computational unreliability, demanding evaluation reforms measuring stability beyond single-sample metrics.

Figures & tables

Figure 1: Main results. (a) Faithfulness components with 0.65 threshold. (b) Implicit vs. explicit CoT depth distributions. (c) Layer-wise activation magnitude with key layers marked. (d) Depth–accuracy relationship. (e) Failure mode distribution. (f) Metric–correctness correlations.

Metric	Implicit	Explicit	$\mathbf{\Delta}$
Accuracy	0.585	0.685	$+0.100$
Reasoning Depth $\mathcal{D}$	0.513	0.523	$+0.010$
Activation Entropy $H$	0.089	0.084	$-0.005$
Reasoning Hops	7.00	7.00	0.00

Table 1: Implicit vs. explicit CoT comparison.

Figure 2: Layer causal importance via noise intervention ( $N=50$ , $\sigma=0.1$ ). Red: positive importance; blue: negative. Middle layers 6–9 show highest causal importance (mean = 0.011).

* $p<0.05$ , ** $p<0.01$
Metric	Pearson $r$	Spearman $\rho$	$p$ -value
Reasoning Depth	0.0317	0.0354	0.4800
Activation Entropy	$-$ 0.0606	$-$ 0.0477	0.1758
Latent Fidelity	$-$ 0.2087	$-$ 0.2054	0.0020
Reasoning Steps	$-$ 0.2699	$-$ 0.2717	0.0000

Table 2: Correlation Between Latent Metrics and Answer Correctness

Figure 3: Detailed analysis. (a) Fidelity vs. correctness with jittered outcomes and trend line. (b) Reasoning depth distribution. (c) Accuracy by difficulty and fidelity level.

Figure 4: Supplementary analyses. (a) Thinking token usage by difficulty. (b) Trajectory similarity distributions with 0.7 support threshold. (c) Ablation study results. (d) Information bottleneck analysis with compression layers marked.

Figure 5: Comparison of Qwen2.5-Math-7B vs. 1.5B. (a) Identical accuracy. (b) Reasoning depth comparison. (c) Multi-metric comparison (normalized).

Metric	7B	1.5B	$\mathbf{\Delta}$
Accuracy	0.610	0.610	0.000
Reasoning Depth $\mathcal{D}$	0.514	0.479	$+0.034$
Activation Entropy $H$	0.090	0.169	$-0.079$
Reasoning Hops	7.00	7.00	0.00

Table 3: Cross-model comparison: 7B vs. 1.5B parameters.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Mean	Std
Accuracy	0.6100	0.4877
Reasoning Depth	0.5139	0.0115
Activation Entropy	0.0895	0.0406
Stability Score	0.6000	0.2000
Alignment Score	0.6869	0.1387
Efficiency Score	0.7369	0.0295

Appendix

Table 4: Latent Reasoning Faithfulness Analysis Results on GSM8K

Silent Failure Rate: 0.0%
Mode	Count	Percentage	Risk
True Positive	0	0.0%	Low
True Negative	195	39.0%	Expected
Silent Failure	0	0.0%	High
Lucky Guess	305	61.0%	Medium
Safety Score: 100.0%

Appendix

Table 5: Failure Mode Distribution (Safety Analysis)

Configuration	Avg Fidelity	Correlation
Full	0.6415	$-$ 0.3146
No Stability	0.7177	$-$ 0.3146
No Alignment	0.6106	$-$ 0.3144
No Efficiency	0.5995	$-$ 0.3110

Appendix

Table 6: Ablation Study on Faithfulness Components

LLM Mathematical Reasoning

Large language model mathematical reasoning studies how AI systems solve, explain, verify, and generalize across mathematical problems and proofs.

See all 270 papers

Understanding and Mitigating Premature Confidence for Better LLM Reasoning
May 23, 2026Jingchu Gai, Guanning Zeng, Christina Baek +4Confidence Estimation in Language Models RL for Language Model Reasoning
1Carnegie Mellon University · 2Tsinghua University
Filtered Reasoning Score: Evaluating Reasoning Quality on a Model's Most-Confident Traces
Apr 13, 2026Manas Pathak, Xingyao Chen, Shuozhe Li +2LLM Evaluation Language Model Generation Evaluation
University of Texas at Austin
A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models
Jul 28, 2026Vivek Shukla, Varun Shukla, Atul +2CoT Faithfulness Mathematical Reasoning Benchmarks
Allenhouse Institute of Technology Kanpur, India

May 23, 2026J/K move · Enter open · S save

Understanding and Mitigating Premature Confidence for Better LLM Reasoning

Jingchu Gai, Guanning Zeng, Christina Baek, Chen Wu+3

1Carnegie Mellon University · 2Tsinghua University

Confidence Estimation in Language Models RL for Language Model Reasoning LLM Uncertainty Estimation LLM Reasoning

Open paper AskPDF arXiv

Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps, limiting the gains from additional test-time compute. Improving reasoning quality directly would require process reward models, but the step-level annotations needed to train them are expensive and scarce. We find such a signal in how the model's confidence evolves during reasoning: premature confidence, the tendency to commit to an answer early and use the remaining tokens to rationalize it, strongly predicts flawed reasoning across tasks and model scales. We exploit this in progressive confidence shaping, a reinforcement learning objective that trains models to update their confidence as they reason rather than commit early -- rewarding gradual confidence growth and penalizing early commitment, with no external labels or reward models. The method improves accuracy and reasoning quality from 1.5B to 8B parameters across arithmetic (Countdown), math (DAPO, AIME), and science (ScienceQA): on Countdown, accuracy improves 3.2x (+42.0pp) and flawed reasoning drops 48pp; on AIME, Pass@64 improves 6.6pp. Consistent with this mechanism, the method also improves faithfulness: on a safety benchmark, our models more transparently surface misleading content in their reasoning traces rather than concealing it. Controlled experiments reveal that the problem and its remedy scale together: premature confidence grows with model size and task difficulty, and so do the gains from addressing it.

Abstract

LLM Mathematical Reasoning

Explore similar work

Understanding and Mitigating Premature Confidence for Better LLM Reasoning

Filtered Reasoning Score: Evaluating Reasoning Quality on a Model's Most-Confident Traces

A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models