As people increasingly rely on large language models (LLMs) for information, model providers need to communicate meaningful estimates of whether individual responses are correct or, more generally, reliable. In the context of hallucinations, existing work uses multicalibration to make such reliability predictions meaningful across application-relevant groups, such as prompt topics. Topic groups alone, however, can conceal systematic errors: within the same topic and confidence level, responses with different chain-of-thought (CoT) properties may exhibit opposing errors that cancel on average. In this work, we introduce reasoning-aware multicalibration, a framework that supplements application-relevant topic groups with groups derived from observable properties of CoT traces. We highlight that, theoretically, groupings based on CoT are "internally important", i.e., groupings for which the multicalibration guarantee is not societally crucial as it is for high-risk topics, but whose inclusion in the multicalibration procedure could easily and significantly improve predictive accuracy of the final reliability predictor. Empirically, we apply our framework to hallucination prediction and unsafe-response detection across a variety of benchmarks, and find that adding CoT groups to topic-based calibration reduces Brier score in both tasks. Interestingly, we find that multicalibrating with respect to CoT groups alone (without any topic information) already improves calibration across topics, suggesting that the predictor's errors depend on how the model reasoned, rather than what it was asked about in the prompt.
Figures & tables
Figure 1: Reliability predictions for LLM responses. Stylized examples of how model providers can use reliability predictions multicalibrated with respect to CoT properties to adapt the presentation of responses to users. Here, responses are visually flagged according to their estimated reliability.
Figure 2: All potential patches until α -multicalibration from Example 3.1 .
Figure 3: Overview of reasoning-aware multicalibration. Topic groups are derived from prompts, and CoT groups from diagnostic questions about reasoning traces. Using labeled calibration data, multicalibration fits corrections to the initial scores.
Figure 4: Constructing CoT groups with diagnostic questions. A practitioner reviews CoTs and defines the diagnostic questions. An LLM annotator answers these questions, producing binary group-membership labels. The example shows a real CoT excerpt from an MMLU question assigned to the computer security embedding group. See Table 3 for the full list of diagnostic questions.
Figure 5: CoT reveals opposing errors within a topic and useful score corrections. (A) Confidence and accuracy for held-out MMLU responses assigned to the computer security embedding group, with topic-only confidence in [0.7,0.8) . (B) Additional held-out Brier reduction from the first CoT correction relative to the topic-only correction at the same predictor.
Condition
MMLU
MMLU-Pro
TriviaQA
PopQA
Uncalibrated
0.233 ± 0.006
0.359 ± 0.007
0.294 ± 0.008
0.382 ± 0.008
Marginal HB
0.137 ± 0.004
0.184 ± 0.004
0.209 ± 0.004
0.148 ± 0.004
Topic-only
0.127 ± 0.003
0.163 ± 0.004
0.185 ± 0.004
0.114 ± 0.004
CoT-only
0.113 ± 0.003
0.141 ± 0.004
0.164 ± 0.004
0.125 ± 0.004
Joint
0.112 ± 0.003
0.139 ± 0.004
0.163 ± 0.004
0.109 ± 0.004
Reduction
11.3%
15.0%
12.4%
4.5%
Table 1: Held-out Brier scores, lower is better. Reduction is Joint’s percentage improvement over Topic-only.
Figure 6: CoT-only calibration improves calibration over topics. The plot shows four MMLU topics. Each point gives mean confidence and accuracy within a topic–confidence bin; size indicates sample count. The diagonal marks agreement between confidence and accuracy.
Figure 7: CoT cues can reveal errors that marginal calibration misses . The plot shows six MMLU topics at confidence of 90–100%. Error is accuracy minus confidence.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: All potential patches (using IGHB patch value) for Example B.1 .
Dataset
Evaluation examples
Answer accuracy
MMLU
4,126
83.28%
MMLU-Pro
3,438
75.36%
TriviaQA
2,938
61.98%
PopQA
4,107
21.09%
Appendix
Table 2: Qwen3-8B answer accuracy and number of held-out evaluation examples for each dataset.
Figure 9: Overlap between embedding-based groups and MMLU subjects. Each cell shows the fraction of examples in a subject assigned to an embedding group. Subjects are ordered by their largest group overlap. Memberships can overlap, so a column need not sum to one. Embedding clusters are named using posterior-weighted c-TF-IDF keywords and manual inspection of representative questions. Names do not affect group membership or calibration.
CoT property
Diagnostic question
Yes
Explicit uncertainty
Does the CoT explicitly express uncertainty about its final answer or about a fact needed to choose the answer?
7,723
Explicit certainty
Does the CoT explicitly state high confidence or certainty about its final answer or a key premise? Do not count merely calling an option “correct.”
420
Knowledge gap or guess
Does the CoT say that it does not know, only weakly remembers, assumes, or guesses a fact needed to answer?
5,195
Multiple plausible answers
Does the CoT seriously treat at least two distinct answers or interpretations as plausible, giving some reason for each? Do not count routine enumeration of options.
6,566
Reasoned elimination
Does the CoT eliminate at least one candidate answer using an explicit reason?
7,479
Self-correction
Does the CoT retract or materially revise an earlier claim, calculation, inference, or tentative answer? Do not count discourse markers such as “wait” without a revision.
3,940
Appendix
Table 3: Diagnostic questions used to construct reasoning-aware groups. Each question is answered Yes or No based on the observed CoT. Yes counts are pooled over the 14,609 evaluation traces across MMLU, MMLU-Pro, TriviaQA, and PopQA; groups overlap. The “Yes” column reports the distribution of positive answers across the questions.
Condition
MMLU
MMLU-Pro
TriviaQA
PopQA
Brier score ↓
Uncalibrated
0.233 ± 0.006
0.359 ± 0.007
0.294 ± 0.008
0.382 ± 0.008
Marginal HB
0.137 ± 0.004
0.184 ± 0.004
0.209 ± 0.004
0.148 ± 0.004
Topic-only
0.127 ± 0.003
0.163 ± 0.004
0.185 ± 0.004
0.114 ± 0.004
CoT-only (diagnostics)
0.113 ± 0.003
0.141 ± 0.004
0.164 ± 0.004
0.125 ± 0.004
CoT-only (text features)
0.118 ± 0.003
0.157 ± 0.003
0.175 ± 0.004
0.132 ± 0.004
Appendix
Table 4: Brier scores for diagnostic and text-feature CoT groups using IGHB, with shared scores, examples, splits, and topic groups.
Figure 10: CoT-associated errors after marginal histogram binning. Each panel identifies a diagnostic, and each horizontal row within a panel is one topic and initial-score bin. Prediction error is accuracy minus mean predicted confidence, in percentage points. Each subgroup must have at least 20 fitting and 20 evaluation examples, otherwise we mark the panel as “insufficient data”.
Condition
MMLU
MMLU-Pro
TriviaQA
PopQA
Mean topic gASCE ↓
Uncalibrated
0.106 ± 0.005
0.216 ± 0.008
0.115 ± 0.007
0.274 ± 0.009
Marginal HB
0.011 ± 0.001
0.027 ± 0.002
0.022 ± 0.002
0.041 ± 0.003
Topic-only
0.008 ± 0.001
0.015 ± 0.002
0.014 ± 0.002
0.009 ± 0.002
CoT-only
0.010 ± 0.001
0.018 ± 0.002
0.017 ± 0.002
0.032 ± 0.002
Joint
0.010 ± 0.001
0.014 ± 0.002
0.014 ± 0.002
0.012 ± 0.001
Appendix
Table 5: Mean and worst-topic calibration error (gASCE) on held-out examples across four datasets. Lower values indicate better topic-level calibration.
Figure 11: Topic-level reliability for all 39 MMLU topics, before calibration and after topic-only or CoT-only IGHB. Each point shows mean confidence and accuracy in a nonempty topic–confidence bin, using ten equal-width bins. Color identifies the topic and point size indicates sample count. The predictors and evaluation examples are the same as in Figure 6 .
Figure 12: CoT-group reliability on held-out MMLU responses, before calibration and after topic-only or CoT-only IGHB. This reverses the audit in Figure 11 : predictions are evaluated within CoT groups rather than topic groups. Each color denotes the Yes-group of one diagnostic question.
Figure 13: CoT subgroup errors after topic-only IGHB calibration on held-out MMLU responses from Qwen3-8B. Columns show all 39 embedding-derived topic groups, using the same names as Figure 9 . Rows show diagnostic questions, with Yes answers in the top panel and No answers in the bottom panel. The pooled topic error is repeated in the first row. Colors show mean predicted correctness minus observed correctness, in percentage points (pp); positive values indicate overprediction. Gray cells contain fewer than 20 responses.
Calibration
Labeler
MMLU
MMLU-Pro
TriviaQA
PopQA
Brier score ↓
Topic-only
—
0.127 ± 0.003
0.163 ± 0.004
0.185 ± 0.004
0.114 ± 0.004
CoT-only
Qwen3-4B
0.127 ± 0.003
0.163 ± 0.004
0.183 ± 0.004
0.129 ± 0.004
CoT-only
Qwen3-8B
0.123 ± 0.003
0.163 ± 0.004
0.174 ± 0.004
0.126 ± 0.004
CoT-only
Gemma4-31B
0.113 ± 0.003
0.141 ± 0.004
0.164 ± 0.004
0.125 ± 0.004
Joint
Qwen3-4B
0.124 ± 0.003
0.155 ± 0.004
0.178 ± 0.004
0.109 ± 0.004
Appendix
Table 6: Results across diagnostic labelers, audited over topics. Other baselines appear in Table 1 .
Calibration
MMLU
MMLU-Pro
TriviaQA
PopQA
Brier score ↓
Topic-only
0.127 ± 0.003
0.161 ± 0.004
0.181 ± 0.004
0.109 ± 0.004
CoT-only
0.112 ± 0.003
0.141 ± 0.004
0.159 ± 0.004
0.119 ± 0.004
Joint
0.112 ± 0.003
0.141 ± 0.004
0.158 ± 0.004
0.103 ± 0.004
Mean topic gASCE ↓
Topic-only
0.009 ± 0.001
0.015 ± 0.002
0.018 ± 0.002
0.009 ± 0.001
Appendix
Table 7: IGLB calibration on the held-out examples. Topic gASCE uses the same eligible topics and ten confidence bins as Table 5 . Lower is better.
Calibrated on
Raw
HB
Topic-only
CoT-only
Joint
Evaluated on MMLU
MMLU
0.2329
0.1366
0.1267
0.1127
0.1124
MMLU-Pro
0.2329
0.1418
0.1416
0.1177
0.1187
TriviaQA
0.2329
0.1950
0.1854
0.1284
0.1303
PopQA
0.2329
0.4385
0.4322
0.2399
0.2688
Evaluated on MMLU-Pro
Appendix
Table 8: Brier score under benchmark transfer. Rows identify the calibration benchmark; bold labels mark fitting on the target’s separate calibration split. Lower is better.
Calibrated on
Raw
HB
Topic-only
CoT-only
Joint
Evaluated on MMLU
MMLU
0.106
0.011
0.008
0.010
0.010
MMLU-Pro
0.106
0.017
0.023
0.017
0.017
TriviaQA
0.106
0.074
0.066
0.026
0.026
PopQA
0.106
0.322
0.316
0.131
0.158
Evaluated on MMLU-Pro
Appendix
Table 9: Membership-weighted mean gASCE over target-defined topics. Rows identify the calibration benchmark; bold labels mark fitting on the target’s separate calibration split. Lower is better.
Calibration
Brier
Mean topic gASCE
Worst-topic gASCE
Uncalibrated
0.166 ± 0.005
0.047 ± 0.003
0.552 ± 0.056
Topic-only
0.107 ± 0.003
0.011 ± 0.001
0.068 ± 0.017
CoT-only (10)
0.118 ± 0.003
0.019 ± 0.001
0.214 ± 0.038
Joint (10)
0.106 ± 0.003
0.010 ± 0.001
0.062 ± 0.017
CoT-only (15)
0.108 ± 0.003
0.016 ± 0.001
0.177 ± 0.037
Joint (15)
0.102 ± 0.003
0.010 ± 0.001
0.106 ± 0.023
Appendix
Table 10: Brier score and topic calibration for the safety prediction task, with ten or fifteen diagnostics, audited over topics. Values are estimates ± one bootstrap standard error ( Section 5 ). Bold marks the lowest point estimate in each column.
CoT property
Diagnostic question
Yes
Request and response
Does the trace explicitly distinguish the user’s request from the response being evaluated when explaining its safety judgment?
4,399
Discussion and assistance
Does the trace explicitly distinguish discussing or describing a harmful activity from providing practical help to carry it out?
524
Harm mechanism
Does the trace describe a concrete way that the response could cause or facilitate harm? Do not count a bare harm-category name or the word unsafe.
1,250
Refusals and disclaimers
Does the trace assess whether a refusal, warning, or disclaimer changes the safety of the rest of the response? Do not count merely noting that a refusal, warning, or disclaimer is present.
183
Context dependence
Does the trace explicitly make its safety judgment depend on an assumption or condition about intent, authorization, audience, or use context? Do not count merely mentioning context.
352
Appendix
Table 11: Additional safety diagnostics questions. The “Yes” column reports the distribution of positive answers across the questions.
The ability of large language models (LLMs) to express calibrated uncertainty is important for safe deployment. Chain-of-thought (CoT) reasoning is widely used to improve accuracy and reliability, but its effect on calibration is not fully understood. We show that this picture is incomplete: in some settings, increasing the reasoning budget beyond a task-specific threshold can cause models to become systematically overconfident, assigning high confidence to incorrect answers. We call this phenomenon Calibration Drift Under Reasoning (CDUR) and study it both theoretically and empirically. We define reasoning budget B and analyze conditions under which Expected Calibration Error ECE(B) follows a non-monotonic pattern: it first decreases as reasoning corrects errors, then increases as longer reasoning produces internally consistent but incorrect explanations. We propose a Hypothesis Lock-In model based on autoregressive generation to explain this behavior. We evaluate Llama-3.1-8B and Llama-3.3-70B on 47 reasoning-trap questions across four reasoning budgets and three seeds (1,368 API calls; 574 valid responses). The 8B model shows non-monotonic calibration behavior, while results for the 70B model are limited to baseline evaluation and are inconclusive for budget-dependent effects. We introduce CABStop, a calibration-aware stopping rule that halts reasoning when confidence diverges from an auxiliary accuracy estimate. These results suggest that increasing reasoning depth does not always improve reliability and should be monitored carefully.
Prakul Sunil Hiremath, Harshit R. Hiremath
Department of Computer Science and Engineering Visvesvaraya Technological University, Belagavi · Department of Computer Science and Business System SG Balekundri Institute of Technology, Belagavi
Accurate confidence estimation is critical for reliable use of large language models (LLMs). Prior work on LLM calibration largely focuses on response-level confidence, which estimates the correctness of a single generated output. However, this formulation is misaligned with many practical settings where the central question is how likely a model is to solve a query overall. We show that this mismatch results from the stochastic nature of modern LLM decoding, under which single-response correctness fails to reflect underlying model capability. To address this issue, we introduce capability calibration, a new evaluation framework for measuring how well query-level confidence aligns with a model's expected accuracy on individual queries. We formally distinguish capability calibration (CC) from response calibration (RC) and show that the two differ both theoretically and empirically. We further show that CC is better suited than RC to applications like pass@k prediction and inference budget allocation. Finally, we evaluate common confidence estimation methods to understand the practical feasibility of CC.
Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success. Existing methods typically elicit confidence only once: either before thinking or after answering. We argue that confidence in reasoning models is state-dependent: before thinking, confidence should estimate the chance of the model correctly solving the prompt, while after thinking it should predict whether the realized answer is likely to be correct. This distinction determines the appropriate supervision target: prompt-level success should supervise confidence estimates made after seeing the prompt, while individual answer-level correctness should supervise confidence estimates made after answering. We introduce CALIBER (Calibration Before and After Reasoning), which elicits both estimates and supervises each with the target matched to its information state. Under this unified protocol, CALIBER reduces Expected Calibration Error (ECE) by 52.5% over the strongest single-confidence baseline on BigMathDigits for the 7B model, while achieving the best Brier score and AUROC, and remains within 2.1 points of the best accuracy. Further, on a larger 30B model, CALIBER achieves the best ECE on BigMathDigits while remaining competitive in Brier score and AUROC. Out of distribution, it achieves the best ECE and Brier score on GPQA and TriviaQA, and remains competitive on SimpleQA. Ablations further show that this position-target alignment is most beneficial under distribution shift where it consistently reduces calibration error across all out-of-distribution benchmarks.