Large Reasoning Models (LRMs) have recently demonstrated significant improvements in complex reasoning. While quantifying generation uncertainty in LRMs is crucial, traditional methods are often insufficient because they do not provide finite-sample guarantees for reasoning-answer generation. Conformal prediction (CP) stands out as a distribution-free and model-agnostic methodology that constructs statistically rigorous uncertainty sets. However, existing CP methods ignore the logical connection between the reasoning trace and the final answer. Additionally, prior studies fail to interpret the origins of uncertainty coverage for LRMs as they typically overlook the specific training factors driving valid reasoning. Notably, it is challenging to disentangle reasoning quality from answer correctness when quantifying uncertainty, while simultaneously establishing theoretical guarantees for computationally efficient explanation methods. To address these challenges, we first propose a novel methodology that quantifies uncertainty in the reasoning-answer structure with statistical guarantees. Subsequently, we develop a unified example-to-step explanation framework using Shapley values that identifies a provably sufficient subset of training examples and their key reasoning steps to preserve the guarantees. We also provide theoretical analyses of our proposed methods. Extensive experiments on challenging reasoning datasets verify the effectiveness of the proposed methods.
Figures & tables
Figure 1: Empirical loss and efficiency results across datasets.
Figure 2: Visualization of our proposed conformal prediction framework on CLEVR-Math.
Model
Method
Top-1 Shapley value
Success rate
LMM-R1
Random
0.042±0.026
0.000±0.000
Ours
0.433±0.116
0.833±0.167
R1-Onevision
Random
0.021±0.038
0.000±0.000
Ours
0.356±0.058
1.000±0.000
Table 1: Data-level explanations of the uncertainty sets.
Model
Method
Top-1 Shapley value
Success rate
LMM-R1
Random
0.020±0.016
0.000±0.000
Ours
0.739±0.128
0.875±0.125
R1-Onevision
Random
0.006±0.005
0.000±0.000
Ours
0.589±0.119
0.750±0.164
Table 2: Step-level explanations of the uncertainty sets.
Figure 3: Analysis of running time.
Dataset
Level ε
Probability
Prediction set size
CLEVR-Math
0.2
0.875±0.125
2.775±0.821
0.4
0.625±0.183
2.750±0.429
0.6
0.500±0.189
2.575±0.301
0.8
0.500±0.189
2.525±0.285
ScienceQA
0.2
0.875±0.125
6.525±0.360
0.4
0.625±0.183
6.375±0.799
Table 3: Empirical probability and prediction set size of prediction sets on LMM-R1 across different levels ε .
Figure 4: Influence of calibration set size for CoRAP.
Figure 5: Influence of maximum numbers of samples.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Empirical loss and efficiency results across datasets and models.
Figure 7: Influence of fine-tuning methods on ScienceQA.
While Large Reasoning Models (LRMs) excel at complex reasoning, alignment through reinforcement learning often induces systemic overconfidence. In production environments, where logits may be unavailable, robust black-box uncertainty quantification (UQ) is essential for trustworthiness and safety. Focusing on question-answering for LRMs, we show that existing black-box methods, such as paraphrase-based self-consistency and confidence verbalization, offer little to no improvement over simple repeated sampling, suggesting that alignment suppresses useful output variability. We introduce prompt-level relaxation operators that broaden the model's effective output distribution by approximating the effect of an optimal policy obtained with a stronger KL-regularization parameter, hence closer to the reference model. Theoretically, we demonstrate that relaxation improves calibration. We propose Jailbreak for Uncertainty (J4U), a jailbreak-derived technique for UQ that empirically reproduces the behavioral signatures predicted by our relaxation theory. Across 3 datasets and 4 LRMs, including a closed-source production model, J4U's improvement over repeated sampling achieves statistical significance in up to 6 times more LRM-dataset-metric settings than the strongest black-box UQ state-of-the-art baseline we evaluate, with average ECE reductions up to 5 times larger. These results provide a practical tool for UQ in black-box LRM deployment.
Language model (LM) "reasoning", commonly described as Chain-of-Thought or test-time scaling, often improves benchmark performance, but the dynamics underlying this process remain poorly understood. We study these dynamics through the lens of uncertainty quantification by treating the "reasoning" traces, the intermediate token sequences generated by LMs, as evolving model states. We summarize each trace by an uncertainty trace profile: a small set of features describing the shape of the uncertainty signal over its trace, such as its slope and linearity. We find that across five LMs evaluated on GSM8K and ProntoQA, these profiles predict whether a trace yields a correct final answer with AUROC up to 0.807, improving markedly on recent related work. We reach AUROC 0.801 using only the first few hundred tokens of full traces, suggesting that errors can be detected early in the generation. A detailed comparison of correct and incorrect traces further reveals qualitatively distinct uncertainty profiles, with correct traces showing a steeper and less linear decline in uncertainty. Together, the results suggest that our method, grounded in decision-making under uncertainty, provides a principled lens for studying the generative process underlying LM "reasoning".
Nils Grünefeld, Bertram Højer, Philipp Mondorf +5
Data Science Section, IT University of Copenhagen · Pioneer Centre for Artificial Intelligence · MaiNLP, Center for Information and Language Processing, LMU Munich +2
Current answering paradigms for Large Reasoning Models (LRMs) often fail to account for the fact that some questions may lie beyond the model's operational capability boundary, leading to long but unproductive reasoning. In this paper, we study whether LRMs expose early signals predictive of such cases, and whether these signals can be used to mitigate unproductive reasoning. In black-box settings, we find that reasoning expressions contain failure-predictive signals. In white-box settings, we show that the hidden states of the last input token contain information that is predictive of whether a question will not be solved correctly under our evaluation setup. Building on these observations, we propose two test-time monitoring strategies: reasoning expression monitoring and hidden states monitoring, that reduce token usage by 62.7-93.6%, substantially improving efficiency and reliability while largely preserving accuracy.