cs.AIMar 19, 2026
SaveHow Uncertainty Estimation Scales with Sampling in Reasoning Models
Organizations: Institute of Computer Science, University of Tartu
Abstract
Uncertainty estimation is critical for deploying reasoning language models, yet remains poorly understood under extended chain-of-thought reasoning. We study parallel sampling as a fully black-box approach using verbalized confidence and self-consistency. Across three reasoning models and 17 tasks spanning mathematics, STEM, and humanities, we characterize how these signals scale. Both self-consistency and verbalized confidence scale in reasoning models, but self-consistency exhibits lower initial discrimination and lags behind verbalized confidence under moderate sampling. Most uncertainty gains, however, arise from signal combination: with just two samples, a hybrid estimator improves AUROC by up to on average and already outperforms either signal alone even when scaled to much larger budgets, after which returns diminish. These effects are domain-dependent: in mathematics, the native domain of RLVR-style post-training, reasoning models achieve higher uncertainty quality and exhibit both stronger complementarity and faster scaling than in STEM or humanities.
Figures & tables
| Task | # examples |
|---|---|
| Mathematical tasks | |
| Math | 1351 |
| GSM8K | 1319 |
| AIME 2024 & 2025 | 60 |
| STEM tasks (excluding math) | |
| Health | 818 |
Table 1: Task families and constituent tasks used in our evaluation, along with the number of examples per task.
| VC (K=1) | VC (K=2) | VC (K=5) | VC (K=8) | SC (K=2) | SC (K=5) | SC (K=8) | SCVC (K=2) | SCVC (K=5) | SCVC (K=8) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Mathematical tasks | ||||||||||
| Math | 78.7 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 81.1 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 84.5 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 84.9 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 66.1 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 74.1 \mathbin{\scalebox{0.8}{\pm}} 1.1 | 77.6 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 85.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 87.9 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 88.7 \mathbin{\scalebox{0.8}{\pm}} 0.4 |
| GSM8K | 68.7 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 71.9 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 75.9 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 77.5 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 63.3 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 68.2 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 70.7 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 77.7 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 80.8 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 82.4 \mathbin{\scalebox{0.8}{\pm}} 0.5 |
| AIME 2024 & 2025 | 66.4 \mathbin{\scalebox{0.8}{\pm}} 7.6 | 67.2 \mathbin{\scalebox{0.8}{\pm}} 8.2 | 78.8 \mathbin{\scalebox{0.8}{\pm}} 6.8 | 81.7 \mathbin{\scalebox{0.8}{\pm}} 5.5 | 82.6 \mathbin{\scalebox{0.8}{\pm}} 6.1 | 85.9 \mathbin{\scalebox{0.8}{\pm}} 7.9 | 90.0 \mathbin{\scalebox{0.8}{\pm}} 4.9 | 89.5 \mathbin{\scalebox{0.8}{\pm}} 5.5 | 91.9 \mathbin{\scalebox{0.8}{\pm}} 5.1 | 94.1 \mathbin{\scalebox{0.8}{\pm}} 2.7 |
| Average (tasks) | 71.3 \mathbin{\scalebox{0.8}{\pm}} 2.6 | 73.4 \mathbin{\scalebox{0.8}{\pm}} 2.8 | 79.7 \mathbin{\scalebox{0.8}{\pm}} 2.3 | 81.4 \mathbin{\scalebox{0.8}{\pm}} 1.9 | 70.6 \mathbin{\scalebox{0.8}{\pm}} 2.1 | 76.1 \mathbin{\scalebox{0.8}{\pm}} 2.6 | 79.4 \mathbin{\scalebox{0.8}{\pm}} 1.6 | 84.2 \mathbin{\scalebox{0.8}{\pm}} 1.9 | 86.8 \mathbin{\scalebox{0.8}{\pm}} 1.7 | 88.4 \mathbin{\scalebox{0.8}{\pm}} 0.9 |
| STEM tasks (excluding math) | ||||||||||
Table 2: AUROC of verbalized confidence (VC), self-consistency (SC), and their combination (SCVC) across task families at varying sampling budgets . Tasks are primarily from MMLU-Pro (Math, STEM, and Social Sciences/Humanities); the STEM domain also includes GPQA Diamond, which is not part of MMLU-Pro. Rows report per-task AUROC macro-averaged over models, and “Average (tasks)” denotes the mean bootstrap standard deviation macro-averaged over models and tasks within task family (domain).
Figure 1: Direct comparison (AUROC vs. cost across datasets) between extended thinking (gpt-oss-20b-high) and shallow thinking (gpt-oss-20b-low).
Figure 2: AUROC as a function of the SC weight in the hybrid SC+VC signal, shown for , , and . Results are averaged across models and tasks within each domain, with shaded regions indicating 95% confidence intervals. Performance is stable across a wide range of , with degradation only at the extremes corresponding to pure VC or pure SC.
Figure 3: Kendall’s rank correlation between VC and SC as a function of the number of samples macro-averaged across reasoning models and task families. Correlation starts low and increases with sampling depth mirroring describing front-loaded gains of simple addition of the two signals and is consistently lower in mathematics than in STEM and humanities coinciding with RLVR training on math.
| Method | Math | STEM | Humanities |
|---|---|---|---|
| VaEl | 73.24 | 77.15 | 70.04 |
| VeEl | 70.80 | 76.95 | 70.61 |
| EpEl | 69.44 | 76.11 | 71.43 |
| VaJu | 70.35 | 76.24 | 71.78 |
| VeJu | 73.82 | 76.37 | 71.49 |
| EpJu | 67.59 | 72.00 | 67.97 |
Table 3: Domain-averaged AUROC for six uncertainty elicitation methods. Math includes AIME, GSM8K, and MMLU-Pro Math; STEM includes Physics, Health, and GPQA; Humanities includes remaining MMLU-Pro tasks.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| VC (K=1) | VC (K=2) | VC (K=5) | VC (K=8) | SC (K=2) | SC (K=5) | SC (K=8) | SCVC (K=2) | SCVC (K=5) | SCVC (K=8) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Math | ||||||||||
| gpt-oss-20b | 80.9 \mathbin{\scalebox{0.8}{\pm}} 2.1 | 83.2 \mathbin{\scalebox{0.8}{\pm}} 1.9 | 87.9 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 88.6 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 63.6 \mathbin{\scalebox{0.8}{\pm}} 2.1 | 71.5 \mathbin{\scalebox{0.8}{\pm}} 2.0 | 75.2 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 87.1 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 89.4 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 90.0 \mathbin{\scalebox{0.8}{\pm}} 0.8 |
| Qwen3-30B-A3B | 76.3 \mathbin{\scalebox{0.8}{\pm}} 1.7 | 79.4 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 81.8 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 81.6 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 62.8 \mathbin{\scalebox{0.8}{\pm}} 2.1 | 70.6 \mathbin{\scalebox{0.8}{\pm}} 1.9 | 74.1 \mathbin{\scalebox{0.8}{\pm}} 1.7 | 82.2 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 85.2 \mathbin{\scalebox{0.8}{\pm}} 1.1 | 86.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 |
| DeepSeek-R1-8B | 78.9 \mathbin{\scalebox{0.8}{\pm}} 1.8 | 80.7 \mathbin{\scalebox{0.8}{\pm}} 1.7 | 83.8 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 84.5 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 71.8 \mathbin{\scalebox{0.8}{\pm}} 2.0 | 80.1 \mathbin{\scalebox{0.8}{\pm}} 1.6 | 83.4 \mathbin{\scalebox{0.8}{\pm}} 1.4 | 87.2 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 89.0 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 89.8 \mathbin{\scalebox{0.8}{\pm}} 0.7 |
| Average (models) | 78.7 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 81.1 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 84.5 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 84.9 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 66.1 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 74.1 \mathbin{\scalebox{0.8}{\pm}} 1.1 | 77.6 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 85.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 87.9 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 88.7 \mathbin{\scalebox{0.8}{\pm}} 0.4 |
| GSM8K | ||||||||||
Table 4: AUROC of verbalized confidence (VC), self-consistency (SC), and their combination (SCVC) across mathematics tasks at different sampling budgets . Rows report per-model results, with the “Average (models)” row denoting the mean bootstrap standard deviation after averaging across models.
| VC (K=1) | VC (K=2) | VC (K=5) | VC (K=8) | SC (K=2) | SC (K=5) | SC (K=8) | SCVC (K=2) | SCVC (K=5) | SCVC (K=8) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Health | ||||||||||
| gpt-oss-20b | 74.3 \mathbin{\scalebox{0.8}{\pm}} 1.4 | 76.4 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 78.6 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 79.1 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 64.4 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 71.2 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 73.9 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 79.8 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 80.3 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 80.6 \mathbin{\scalebox{0.8}{\pm}} 0.7 |
| Qwen3-30B-A3B | 67.4 \mathbin{\scalebox{0.8}{\pm}} 1.6 | 69.7 \mathbin{\scalebox{0.8}{\pm}} 1.4 | 72.8 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 73.3 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 62.1 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 68.7 \mathbin{\scalebox{0.8}{\pm}} 1.1 | 71.0 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 74.3 \mathbin{\scalebox{0.8}{\pm}} 1.4 | 76.7 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 77.4 \mathbin{\scalebox{0.8}{\pm}} 0.7 |
| DeepSeek-R1-8B | 67.9 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 69.7 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 70.1 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 70.4 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 63.6 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 68.5 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 70.4 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 74.6 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 75.2 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 75.6 \mathbin{\scalebox{0.8}{\pm}} 0.7 |
| Average (models) | 69.9 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 71.9 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 73.8 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 74.3 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 63.4 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 69.5 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 71.8 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 76.2 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 77.4 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 77.9 \mathbin{\scalebox{0.8}{\pm}} 0.4 |
| Biology | ||||||||||
Table 5: AUROC of verbalized confidence (VC), self-consistency (SC), and their combination (SCVC) across STEM tasks at different sampling budgets . Rows report per-model results, with the “Average (models)” row denoting the mean bootstrap standard deviation after averaging across models..
| VC (K=1) | VC (K=2) | VC (K=5) | VC (K=8) | SC (K=2) | SC (K=5) | SC (K=8) | SCVC (K=2) | SCVC (K=5) | SCVC (K=8) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Psychology | ||||||||||
| gpt-oss-20b | 70.0 \mathbin{\scalebox{0.8}{\pm}} 1.7 | 72.3 \mathbin{\scalebox{0.8}{\pm}} 1.7 | 75.9 \mathbin{\scalebox{0.8}{\pm}} 1.2 | 77.0 \mathbin{\scalebox{0.8}{\pm}} 1.1 | 66.0 \mathbin{\scalebox{0.8}{\pm}} 1.4 | 73.2 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 75.8 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 78.2 \mathbin{\scalebox{0.8}{\pm}} 1.4 | 80.0 \mathbin{\scalebox{0.8}{\pm}} 1.1 | 80.8 \mathbin{\scalebox{0.8}{\pm}} 1.0 |
| Qwen3-30B-A3B | 65.2 \mathbin{\scalebox{0.8}{\pm}} 1.8 | 68.1 \mathbin{\scalebox{0.8}{\pm}} 1.6 | 71.3 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 72.1 \mathbin{\scalebox{0.8}{\pm}} 1.1 | 60.6 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 67.3 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 69.8 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 72.9 \mathbin{\scalebox{0.8}{\pm}} 1.6 | 76.7 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 77.9 \mathbin{\scalebox{0.8}{\pm}} 1.1 |
| DeepSeek-R1-8B | 68.9 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 70.8 \mathbin{\scalebox{0.8}{\pm}} 1.4 | 71.6 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 71.9 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 63.2 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 69.5 \mathbin{\scalebox{0.8}{\pm}} 1.5 | 71.9 \mathbin{\scalebox{0.8}{\pm}} 1.4 | 75.3 \mathbin{\scalebox{0.8}{\pm}} 1.3 | 76.5 \mathbin{\scalebox{0.8}{\pm}} 1.1 | 77.1 \mathbin{\scalebox{0.8}{\pm}} 1.0 |
| Average (models) | 68.1 \mathbin{\scalebox{0.8}{\pm}} 1.0 | 70.4 \mathbin{\scalebox{0.8}{\pm}} 0.9 | 72.9 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 73.7 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 63.3 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 70.0 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 72.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 75.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 77.7 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 78.6 \mathbin{\scalebox{0.8}{\pm}} 0.6 |
| Law | ||||||||||
Table 6: AUROC of verbalized confidence (VC), self-consistency (SC), and their combination (SCVC) across humanities tasks at different sampling budgets . Rows report per-model results, with the “Average (models)” row denoting the mean bootstrap standard deviation after averaging across models.
| VC (K=1) | VC (K=2) | VC (K=5) | VC (K=8) | SC (K=2) | SC (K=5) | SC (K=8) | SCVC (K=2) | SCVC (K=5) | SCVC (K=8) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Math | ||||||||||
| gpt-oss-20b | 94.1 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.1 |
| Qwen3-30B-A3B | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.7 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.7 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.7 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.7 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 94.5 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.7 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.7 \mathbin{\scalebox{0.8}{\pm}} 0.1 |
| DeepSeek-R1-8B | 91.8 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 91.8 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 92.8 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 92.9 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 91.8 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 92.8 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 92.9 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 91.8 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 92.8 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 92.9 \mathbin{\scalebox{0.8}{\pm}} 0.2 |
| Average (models) | 93.5 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 93.4 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 93.4 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 93.4 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.1 | 94.0 \mathbin{\scalebox{0.8}{\pm}} 0.1 |
| GSM8K | ||||||||||
Table 7: Accuracy of verbalized confidence (VC), self-consistency (SC), and their combination (SCVC) across mathematics tasks at different sampling budgets . Rows report per-model results, with the “Average (models)” row denoting the mean bootstrap standard deviation after averaging across models.
| VC (K=1) | VC (K=2) | VC (K=5) | VC (K=8) | SC (K=2) | SC (K=5) | SC (K=8) | SCVC (K=2) | SCVC (K=5) | SCVC (K=8) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Health | ||||||||||
| gpt-oss-20b | 74.2 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 74.2 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 75.6 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 75.8 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 74.2 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 75.6 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 75.8 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 74.2 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 75.6 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 75.8 \mathbin{\scalebox{0.8}{\pm}} 0.4 |
| Qwen3-30B-A3B | 75.4 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 75.4 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 76.4 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 76.5 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 75.4 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 76.4 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 76.5 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 75.4 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 76.4 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 76.5 \mathbin{\scalebox{0.8}{\pm}} 0.3 |
| DeepSeek-R1-8B | 70.0 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 70.0 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 72.0 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 72.2 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 70.0 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 72.0 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 72.2 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 70.0 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 72.0 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 72.2 \mathbin{\scalebox{0.8}{\pm}} 0.4 |
| Average (models) | 73.2 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 73.2 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 74.7 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 74.8 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 73.2 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 74.7 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 74.8 \mathbin{\scalebox{0.8}{\pm}} 0.2 | 73.2 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 74.7 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 74.8 \mathbin{\scalebox{0.8}{\pm}} 0.2 |
| Biology | ||||||||||
Table 8: Accuracy of verbalized confidence (VC), self-consistency (SC), and their combination (SCVC) across STEM tasks at different sampling budgets . Rows report per-model results, with the “Average (models)” row denoting the mean bootstrap standard deviation after averaging across models.
| VC (K=1) | VC (K=2) | VC (K=5) | VC (K=8) | SC (K=2) | SC (K=5) | SC (K=8) | SCVC (K=2) | SCVC (K=5) | SCVC (K=8) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Psychology | ||||||||||
| gpt-oss-20b | 71.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 71.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 73.2 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 73.4 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 71.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 73.2 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 73.4 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 71.5 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 73.2 \mathbin{\scalebox{0.8}{\pm}} 0.7 | 73.4 \mathbin{\scalebox{0.8}{\pm}} 0.6 |
| Qwen3-30B-A3B | 76.9 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 76.8 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 77.5 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 77.5 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 76.8 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 77.5 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 77.5 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 76.8 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 77.5 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 77.5 \mathbin{\scalebox{0.8}{\pm}} 0.4 |
| DeepSeek-R1-8B | 73.3 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 73.3 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 74.7 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 75.0 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 73.3 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 74.7 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 75.0 \mathbin{\scalebox{0.8}{\pm}} 0.5 | 73.3 \mathbin{\scalebox{0.8}{\pm}} 0.8 | 74.7 \mathbin{\scalebox{0.8}{\pm}} 0.6 | 75.0 \mathbin{\scalebox{0.8}{\pm}} 0.5 |
| Average (models) | 73.9 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 73.9 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 75.1 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 75.3 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 73.9 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 75.1 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 75.3 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 73.9 \mathbin{\scalebox{0.8}{\pm}} 0.4 | 75.1 \mathbin{\scalebox{0.8}{\pm}} 0.3 | 75.3 \mathbin{\scalebox{0.8}{\pm}} 0.3 |
| Law | ||||||||||
Table 9: Accuracy of verbalized confidence (VC), self-consistency (SC), and their combination (SCVC) across humanities tasks at different sampling budgets . Rows report per-model results, with the “Average (models)” row denoting the mean bootstrap standard deviation after averaging across models.
Figure 4: Overview of uncertainty instructions prompts defining VC methods. (a) Vanilla uncertainty instruction, (b) Verification uncertainty instruction and (c) epistemic uncertainty instruction. Each of the instructions is used both for elicitation and judge methods. For judge method, the epistemic uncertainty instructions are a bit different, as it needs to pay attention to the solver’s reasoning trace, not its own.