cs.AIMar 11, 2026

Verbalizing LLM's Higher-order Uncertainty via Imprecise Probabilities

Authors: Anita Yang, Krikamol Muandet, Michele Caprio, Siu Lun Chau, Masaki Adachi

Organizations: Lattice Lab, Toyota Motor Corporation, Japan · Department of Computer Science, University of Tokyo, Japan · Rational Intelligence Lab, CISPA, Helmholtz Center for Information Security, Germany · Department of Computer Science, University of Manchester, United Kingdom · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Despite the growing demand for eliciting uncertainty from large language models (LLMs), empirical evidence suggests that LLM behavior is not always adequately captured by the elicitation techniques developed under the classical probabilistic uncertainty framework. This mismatch leads to systematic failure modes, particularly in settings that involve ambiguous question-answering, in-context learning, and self-reflection. To address this, we propose novel prompt-based uncertainty elicitation techniques grounded in imprecise probabilities, a principled framework for repesenting and eliciting higher-order uncertainty. Here, first-order uncertainty captures uncertainty over possible responses to a prompt, while second-order uncertainty (uncertainty about uncertainty) quantifies indeterminacy in the underlying probability model itself. We introduce general-purpose prompting and post-processing procedures to directly elicit and quantify both orders of uncertainty, and demonstrate their effectiveness across diverse settings. Our approach enables more faithful uncertainty reporting from LLMs, improving credibility and supporting downstream decision-making.

Figures & tables

Figure 1: Collection of failure modes in prior verbalized uncertainty scores. (a) Example of an ambiguous question. (b) The prior uncertainty score fails to distinguish between clear and ambiguous question distributions. (c) The prior score also fails to track the decrease in prediction error, which should reflect reduced uncertainty as more in-context examples are provided. (d) Self-reflection on answer-wise probability/uncertainty should explain the rationale behind answer selection, but it often fails to do so.
Figure 2: Our imprecise probabilities–based approach. (a) Classical precise probability provides point estimates. (b) Imprecise probabilities instead represents uncertainty as intervals. (c) This enables more reasonable elicitation of question ambiguity. (d) It more closely tracks predictive error. (e) It aligns the LLM’s answer selection with the selection implied by its imprecise probability estimates.
Figure 3
Figure 4
Figure 3: Learnable vs. noisy transforms.
Figure 4: (a) For first-order uncertainty estimation, both vanilla and De Finetti capture the underlying ambiguity noise pp , but (b) for second-order, our methods stay flat, supporting the disentanglement of uncertainty source.
Figure 5: (a) For individual EU elicitation, vanilla incorrectly increases uncertainty, whereas ProbInt-EU tracks prediction error. (b) For group EU elicitation, our Credal-EU substantially improves AUROC.
 GPT-5gemini-2.5-pro 
MethodMMLU-ProNon-MAQANon-AmbigQAMMLU-ProNon-MAQANon-AmbigQAAvg. Rank
Is true prob.0.6850 ±\pm 0.02380.5835 ±\pm 0.00410.5584 ±\pm 0.01570.6583 ±\pm 0.01970.6296 ±\pm 0.00680.5897 ±\pm 0.00578.67
Label prob.0.7649 ±\pm 0.02200.6430 ±\pm 0.00620.5876 ±\pm 0.01760.6377 ±\pm 0.01530.6483 ±\pm 0.01260.5582 ±\pm 0.03197.83
MI-Clarifications0.7790 ±\pm 0.01140.6528 ±\pm 0.00800.5840 ±\pm 0.01070.6348 ±\pm 0.01440.6546 ±\pm 0.01040.5497 ±\pm 0.02677.83
DiNCO0.7087 ±\pm 0.02140.5642 ±\pm 0.00660.5219 ±\pm 0.00350.7133 ±\pm 0.01670.5749 ±\pm 0.02240.5275 ±\pm 0.01909.17
Vanilla0.8587 ±\pm 0.01920.7577 ±\pm 0.01940.7756 ±\pm 0.01380.7326 ±\pm 0.02210.6889 ±\pm 0.02630.6688 ±\pm 0.02094.17
Table 1: AUROC for correctness detection without ambiguity. Best result bolded in blue , second best in orange .
Figure 6: Credal-set from LLM ensemble: model disagreement induces lower/upper probability bounds for each output, which are used to compute MMI.
Figure 7: AUROC for ambiguity detection.
Figure 8: AUROC for correctness with ambiguity.
Figure 9: API cost on MMLU-Pro across all methods.
Figure 10: ProbInt more closely matches the KL metric.
Figure 11: Concordance index ( ↑\uparrow better) between uncertainty scores and the KL metric.
Figure 12: The LLM’s prediction y^LLM\hat{y}_{\text{LLM}} aligns with the IP-rational decision.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 16
Figure 17
Figure 18
Figure 19
Figure 15: First- and second-order uncertainty can be computed either for the (i) correctness of a specific answer yy , or (ii) over the candidate answers set Y\mathcal{Y}
Figure 16: Approximations of second-order uncertainty with fixed first-order noise (lower-case probability p=0.25p=0.25 ) (with moving average window of 10). Both Pos and Credal generally match shape of the error.
Figure 22
Figure 23
Figure 24
Figure 17: Second-order uncertainty estimates elicited by variants of the ProbInt prompt consistently track the overall error trend.
Figure 18: DeFinetti best aligns with the proxy for first-order uncertainty
More in this topic

Imprecise Probability

See all 9 papers

Explore similar work

CardsList

  1. The Anatomy of Uncertainty in LLMs

    Mar 26, 2026Aditya Taparia, Ransalu Senanayake, Kowshik Thopalli +1LLM AuditingLLM Reliability

    School of Computing and Augmented Intelligence, Arizona State University · Lawrence Livermore National Laboratory

  2. LLMs Should Express Uncertainty Explicitly

    Apr 7, 2026Junyu Guo, Shangding Gu, Ming Jin +2Language Model Self-AssessmentLLM Uncertainty Estimation

    University of California, Berkeley · Virginia Tech

  3. Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation

    Mar 6, 2026Changcheng Li, Jiancan Wu, Hengheng Zhang +5Confidence Estimation in Language ModelsLLM Uncertainty Estimation

    University of Science and Technology of China · Huawei Inc.