Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models

Toghrul Abbasli; Kentaroh Toyoda; Yuan Wang; Li Chen

cs.CLAug 23, 2026

Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models

Authors: Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Li Chen

Organizations: Tsinghua University, Haidian District, Beijing, 100084, P. R. China · Vulcan Research, AIFT, Singapore · Keio Global Research Institute (KGRI), Japan · China Mobile Research Institute, China ·

Abstract

Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is misaligned with factual correctness. Response-level confidence is a coarse signal: a single generation can mix correct and incorrect statements, so a single number is not actionable for users that must accept, reject, or verify individual pieces of information. We study claim-level confidence calibration as a decision-relevant uncertainty signal: each response is decomposed into atomic, verifiable claims, and each claim is assigned a calibrated confidence using inference-time signals from consistency across samples and self-verification. Our framework operates in closed-box settings (no logits, no fine-tuning) and applies post-hoc calibration directly at the claim level, enabling selective intervention such as evidence retrieval or human review for low-confidence claims. Across TriviaQA and TruthfulQA we evaluate seven baselines on six recent models (Llama-3.1, Mistral, Qwen2.5, DeepSeek-R1, GPT-4, GPT-4o), and show that claim-level decomposition combined with post-hoc calibration reduces expected calibration error on factual questions while exposing failure modes on adversarial false-premise questions where decision-makers most need reliable uncertainty estimates.

Figures & tables

Figure 1. Proposed pipeline for claim-level confidence calibration. Given a question

x

, we (i) sample

N

candidate responses via greedy + stochastic decoding, (ii) decompose the primary response into atomic claims

\{c_{1},\dots,c_{k}\}

, and (iii) score each claim with two complementary signals—verification

\operatorname{ver}(c_{i})\in[0,1]

from self-evaluation and consistency

\operatorname{freq}(c_{i})=\tfrac{1}{N}\sum_{j}\mathbf{1}[y^{(j)}\!\models c_{i}]

from cross-sample support. The two signals are combined via a convex mixture

\hat{p}(c_{i})=\alpha\,\operatorname{freq}(c_{i})+(1{-}\alpha)\,\operatorname{ver}(c_{i})

with

\alpha^{*}

tuned on held-out validation, and thresholded at

\tau

to retain or flag each claim. A horizontal pipeline diagram with six color-coded stages flowing left to right: (1) a blue Question node showing an example query; (2) a steel-blue In-context reasoning node producing one greedy and N-1 stochastic samples, with three example sampled responses shown to the right; (3) a green Claim extraction node decomposing the primary response into atomic claims, with three example claims shown to the right; (4) two parallel branches—an orange Verification (ver) branch producing per-claim probabilities, and a mauve Consistency (freq) branch counting verifier-confirmed support across the N samples; (5) a purple Calibrated mixing node combining the two signals as a convex combination with weight alpha; (6) a crimson Per-claim confidence output node applying a retention threshold tau. A bottom legend explains each color's role and notes that verification is strong when the model self-assesses well, while consistency is strong on adversarial or false-premise inputs.

Models & Baselines	TriviaQA							TruthfulQA
	↓ECE	↓smECE	↓BS	↑AUROC	↓AURC	BLEU	$\alpha$	↓ECE	↓smECE	↓BS	↑AUROC	↓AURC	BLEU	$\alpha$
Llama-3.1-8B-Instruct
Few-shot	0.046	0.040	0.286	0.532	0.523	0.041	–	0.386	0.333	0.165	0.794	0.001	0.190	–
Few-shot CoT	0.087	0.065	0.273	0.539	0.436	0.033	–	0.346	0.309	0.136	0.569	0.006	0.191	–
Verbal. CoT	0.113	0.064	0.293	0.564	0.413	0.033	–	0.499	0.348	0.349	0.161	0.005	0.191	–
PS	0.013	0.013	0.249	0.495	0.523	0.041	–	0.363	0.319	0.134	0.794	0.001	0.190	–

Table 1. Evaluation results for Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, Qwen2.5-7B-Instruct, DeepSeek-R1, GPT-4, and GPT-4o on TriviaQA and TruthfulQA. “Ours” rows report the test-set values at the per-model best

\alpha

tuned on a held-out split (or

\alpha{=}0

when no held-out tuning was available). A comprehensive results table showing calibration metrics (ECE, smECE, BS, AUROC, AURC, BLEU) for six language models across two datasets (TriviaQA and TruthfulQA). Rows are grouped by model, with baselines including Few-shot, Few-shot CoT, Verbalized CoT, PS, TS, LMvsLM, HalluMeasure, and the proposed method. Bold values indicate best performance per model and dataset. The proposed method is highlighted in yellow.

Panel of Figure 2 — Figure 2. Reliability diagrams (10 bins) for GPT-4o on TriviaQA. Bar colors and percentages indicate the proportion of points per bin. Additional models, datasets, and baselines are in Appendix D .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Open-box?	Claim-level?	Consistency?	Calibration?
Fine-tuning ( 19 )	Yes	No	No	Yes
TS/PS ( 12 )	Logits	No	No	Yes
Verbalized ( 35 )	No	No	No	Limited
Self-consistency ( 40 )	No	No	Yes	No
SelfCheckGPT ( 30 )	No	No	Yes	No
LMvsLM ( 5 )	No	Yes	No	No

Appendix

Table 2. Comparison of uncertainty estimation methods for LLMs.

Expression	Score
Very low	0.0
Low	0.3
Somewhat low	0.45
Medium	0.5
Somewhat high	0.65
High	0.7

Appendix

Table 3. Mapping for qualitative confidence expressions and equivalent probability score for LLM.

Method	API calls/query	Relative cost
Few-shot / CoT	1	$1\times$
TS / PS	1 + calibration	$\sim\!1\times$
Verbalized CoT	1	$1\times$
HalluMeasure	1 + NLI $\times$ $n_{c}$	$\sim\!5\times$
LMvsLM	5 rounds $\times$ 2	$\sim\!10\times$
Ours	1 + 1 extract + $n_{c}$ verify	$\sim\!6\times$

Appendix

Table 4. Approximate inference cost per query in number of LLM API calls.

Dataset	Avg. claims/resp.	Range
TriviaQA	3–5	1–8
TruthfulQA	4–6	2–10

Appendix

Table 5. Average claims extracted per response.

Model / Dataset	Best $\alpha$	Held-out ECE
GPT-4o / TriviaQA	1.0	0.022
DeepSeek-R1 / TruthfulQA	1.0	0.096
Llama-3.1 / TriviaQA	0.8	0.139
Qwen2.5 / TriviaQA	0.7	0.022

Appendix

Table 6.

\alpha

sensitivity (held-out ECE) and best

\alpha

per configuration.

Panel of Figure 3 — Figure 3. Reliability diagrams for calibration methods using GPT-4o on TriviaQA. The number of diagram bins is 10. The color as well as the percentage number within each bar indicate the proportion of total points contained in each bin.

Panel of Figure 4 — Figure 4. Reliability diagrams for calibration methods with 10 bins using DeepSeek-R1 on TriviaQA and TruthfulQA. The number of diagram bins is 10. The color and the percentage number on each bar indicate the proportion of total points contained in each bin.

Panel of Figure 5 — Figure 5. Reliability diagrams for calibration methods with 10 bins using Llama 3.1 8B on TriviaQA. The color and the percentage number on each bar indicate the proportion of total points contained in each bin. A grid of seven reliability diagrams showing confidence calibration for Llama-3.1-8B-Instruct on TriviaQA. Each subplot displays predicted confidence versus actual accuracy with a diagonal reference line for perfect calibration. Bars are colored and labeled with percentages indicating the proportion of data points in each of 10 bins. Methods shown include sequence likelihood, CoT sequence likelihood, verbalized CoT, Platt scaling, temperature scaling, LMvsLM, and HalluMeasure.

Panel of Figure 6 — Figure 6. Reliability diagrams for calibration methods with 10 bins using Mistral v0.3 7B on TriviaQA. The color and the percentage number within each bar indicate the proportion of total points contained in each bin.

Panel of Figure 7 — Figure 7. Reliability diagrams for calibration methods using Qwen2.5 7B on TriviaQA. The number of diagram bins is 10. The color and the percentage number within each bar indicate the proportion of total points contained in each bin.

Panel of Figure 8 — Figure 8. Reliability diagrams for different calibration methods with 10 bins using Llama 3.1 8B on TruthfulQA. The number of diagram bins is 10. The color and the percentage number on each bar indicate the proportion of total points contained in each bin.

Panel of Figure 9 — Figure 9. Reliability diagrams for different calibration methods with 10 bins using Mistral v0.3 7B on TruthfulQA. The number of diagram bins is 10. The color and the percentage number on each bar indicate the proportion of total points contained in each bin.

Panel of Figure 10 — Figure 10. Reliability diagrams for calibration methods using Qwen2.5 7B on TruthfulQA. The number of diagram bins is 10. The color and the percentage number within each bar indicate the proportion of total points contained in each bin.

Panel of Figure 11 — Figure 11. Reliability diagrams for calibration methods using GPT-4o on TruthfulQA. The number of diagram bins is 10. The color as well as the percentage number within each bar indicate the proportion of total points contained in each bin.

Confidence Estimation in Language Models

Confidence estimation in language models measures whether a model’s uncertainty signals predict answer correctness and support reliable decisions.

See all 229 papers

On Calibration of Large Language Models: From Response To Capability
Feb 14, 2026Sin-Han Yang, Cheng-Kuang Wu, Chieh-Yen Lin +3LLM Evaluation Confidence Estimation in Language Models
Appier AI Research · National Taiwan University
Know When You're Wrong: Aligning Confidence with Correctness for LLM Error Detection
Feb 18, 2026Xie Xiaohu, Liu Xiaohu, Yao BenjaminLanguage Model Error Detection Confidence Estimation in Language Models
Alexa AI, Amazon, Seattle, WA, USA
Closing the Confidence-Faithfulness Gap in Large Language Models
Mar 26, 2026Miranda Muqing Miao, Lyle UngarConfidence Estimation in Language Models Verbalized Confidence Estimation
University of Pennsylvania · Department of Computer and Information Science

Feb 14, 2026J/K move · Enter open · S save

On Calibration of Large Language Models: From Response To Capability

Sin-Han Yang, Cheng-Kuang Wu, Chieh-Yen Lin, Yun-Nung Chen+2

Appier AI Research · National Taiwan University

LLM Evaluation Confidence Estimation in Language Models Language Model Calibration LLM Reliability LLM Uncertainty Estimation Confidence Estimation

Open paper AskPDF arXiv

Accurate confidence estimation is critical for reliable use of large language models (LLMs). Prior work on LLM calibration largely focuses on response-level confidence, which estimates the correctness of a single generated output. However, this formulation is misaligned with many practical settings where the central question is how likely a model is to solve a query overall. We show that this mismatch results from the stochastic nature of modern LLM decoding, under which single-response correctness fails to reflect underlying model capability. To address this issue, we introduce capability calibration, a new evaluation framework for measuring how well query-level confidence aligns with a model's expected accuracy on individual queries. We formally distinguish capability calibration (CC) from response calibration (RC) and show that the two differ both theoretically and empirically. We further show that CC is better suited than RC to applications like pass@k prediction and inference budget allocation. Finally, we evaluate common confidence estimation methods to understand the practical feasibility of CC.

Abstract

Confidence Estimation in Language Models

Explore similar work

On Calibration of Large Language Models: From Response To Capability

Know When You're Wrong: Aligning Confidence with Correctness for LLM Error Detection

Closing the Confidence-Faithfulness Gap in Large Language Models