Reasoning-Aware Multicalibrated Reliability Predictors for Language Models

Ekaterina Fedorova; Alessandro Morosini; Chara Podimata

cs.CYOct 19, 2026

Reasoning-Aware Multicalibrated Reliability Predictors for Language Models

Authors: Ekaterina Fedorova, Alessandro Morosini, Chara Podimata

Organizations: Massachusetts Institute of Technology.

Abstract

As people increasingly rely on large language models (LLMs) for information, model providers need to communicate meaningful estimates of whether individual responses are correct or, more generally, reliable. In the context of hallucinations, existing work uses multicalibration to make such reliability predictions meaningful across application-relevant groups, such as prompt topics. Topic groups alone, however, can conceal systematic errors: within the same topic and confidence level, responses with different chain-of-thought (CoT) properties may exhibit opposing errors that cancel on average. In this work, we introduce reasoning-aware multicalibration, a framework that supplements application-relevant topic groups with groups derived from observable properties of CoT traces. We highlight that, theoretically, groupings based on CoT are "internally important", i.e., groupings for which the multicalibration guarantee is not societally crucial as it is for high-risk topics, but whose inclusion in the multicalibration procedure could easily and significantly improve predictive accuracy of the final reliability predictor. Empirically, we apply our framework to hallucination prediction and unsafe-response detection across a variety of benchmarks, and find that adding CoT groups to topic-based calibration reduces Brier score in both tasks. Interestingly, we find that multicalibrating with respect to CoT groups alone (without any topic information) already improves calibration across topics, suggesting that the predictor's errors depend on how the model reasoned, rather than what it was asked about in the prompt.

Figures & tables

Figure 1: Reliability predictions for LLM responses. Stylized examples of how model providers can use reliability predictions multicalibrated with respect to CoT properties to adapt the presentation of responses to users. Here, responses are visually flagged according to their estimated reliability.

Figure 2: All potential patches until

\alpha

-multicalibration from Example 3.1 .

Figure 3: Overview of reasoning-aware multicalibration. Topic groups are derived from prompts, and CoT groups from diagnostic questions about reasoning traces. Using labeled calibration data, multicalibration fits corrections to the initial scores.

Figure 4: Constructing CoT groups with diagnostic questions. A practitioner reviews CoTs and defines the diagnostic questions. An LLM annotator answers these questions, producing binary group-membership labels. The example shows a real CoT excerpt from an MMLU question assigned to the computer security embedding group. See Table 3 for the full list of diagnostic questions.

Figure 5: CoT reveals opposing errors within a topic and useful score corrections. (A) Confidence and accuracy for held-out MMLU responses assigned to the computer security embedding group, with topic-only confidence in

[0.7,0.8)

. (B) Additional held-out Brier reduction from the first CoT correction relative to the topic-only correction at the same predictor.

Condition	MMLU	MMLU-Pro	TriviaQA	PopQA
Uncalibrated	0.233 $\pm$ 0.006	0.359 $\pm$ 0.007	0.294 $\pm$ 0.008	0.382 $\pm$ 0.008
Marginal HB	0.137 $\pm$ 0.004	0.184 $\pm$ 0.004	0.209 $\pm$ 0.004	0.148 $\pm$ 0.004
Topic-only	0.127 $\pm$ 0.003	0.163 $\pm$ 0.004	0.185 $\pm$ 0.004	0.114 $\pm$ 0.004
CoT-only	0.113 $\pm$ 0.003	0.141 $\pm$ 0.004	0.164 $\pm$ 0.004	0.125 $\pm$ 0.004
Joint	0.112 $\pm$ 0.003	0.139 $\pm$ 0.004	0.163 $\pm$ 0.004	0.109 $\pm$ 0.004
Reduction	11.3%	15.0%	12.4%	4.5%

Table 1: Held-out Brier scores, lower is better. Reduction is Joint’s percentage improvement over Topic-only.

Figure 6: CoT-only calibration improves calibration over topics. The plot shows four MMLU topics. Each point gives mean confidence and accuracy within a topic–confidence bin; size indicates sample count. The diagonal marks agreement between confidence and accuracy.

Figure 7: CoT cues can reveal errors that marginal calibration misses . The plot shows six MMLU topics at confidence of 90–100%. Error is accuracy minus confidence.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: All potential patches (using IGHB patch value) for Example B.1 .

Dataset	Evaluation examples	Answer accuracy
MMLU	4,126	83.28%
MMLU-Pro	3,438	75.36%
TriviaQA	2,938	61.98%
PopQA	4,107	21.09%

Appendix

Table 2: Qwen3-8B answer accuracy and number of held-out evaluation examples for each dataset.

Figure 9: Overlap between embedding-based groups and MMLU subjects. Each cell shows the fraction of examples in a subject assigned to an embedding group. Subjects are ordered by their largest group overlap. Memberships can overlap, so a column need not sum to one. Embedding clusters are named using posterior-weighted c-TF-IDF keywords and manual inspection of representative questions. Names do not affect group membership or calibration.

CoT property	Diagnostic question	Yes
Explicit uncertainty	Does the CoT explicitly express uncertainty about its final answer or about a fact needed to choose the answer?	7,723
Explicit certainty	Does the CoT explicitly state high confidence or certainty about its final answer or a key premise? Do not count merely calling an option “correct.”	420
Knowledge gap or guess	Does the CoT say that it does not know, only weakly remembers, assumes, or guesses a fact needed to answer?	5,195
Multiple plausible answers	Does the CoT seriously treat at least two distinct answers or interpretations as plausible, giving some reason for each? Do not count routine enumeration of options.	6,566
Reasoned elimination	Does the CoT eliminate at least one candidate answer using an explicit reason?	7,479
Self-correction	Does the CoT retract or materially revise an earlier claim, calculation, inference, or tentative answer? Do not count discourse markers such as “wait” without a revision.	3,940

Appendix

Table 3: Diagnostic questions used to construct reasoning-aware groups. Each question is answered Yes or No based on the observed CoT. Yes counts are pooled over the 14,609 evaluation traces across MMLU, MMLU-Pro, TriviaQA, and PopQA; groups overlap. The “Yes” column reports the distribution of positive answers across the questions.

Condition	MMLU	MMLU-Pro	TriviaQA	PopQA
Brier score $\downarrow$
Uncalibrated	0.233 $\pm$ 0.006	0.359 $\pm$ 0.007	0.294 $\pm$ 0.008	0.382 $\pm$ 0.008
Marginal HB	0.137 $\pm$ 0.004	0.184 $\pm$ 0.004	0.209 $\pm$ 0.004	0.148 $\pm$ 0.004
Topic-only	0.127 $\pm$ 0.003	0.163 $\pm$ 0.004	0.185 $\pm$ 0.004	0.114 $\pm$ 0.004
CoT-only (diagnostics)	0.113 $\pm$ 0.003	0.141 $\pm$ 0.004	0.164 $\pm$ 0.004	0.125 $\pm$ 0.004
CoT-only (text features)	0.118 $\pm$ 0.003	0.157 $\pm$ 0.003	0.175 $\pm$ 0.004	0.132 $\pm$ 0.004

Appendix

Table 4: Brier scores for diagnostic and text-feature CoT groups using IGHB, with shared scores, examples, splits, and topic groups.

Figure 10: CoT-associated errors after marginal histogram binning. Each panel identifies a diagnostic, and each horizontal row within a panel is one topic and initial-score bin. Prediction error is accuracy minus mean predicted confidence, in percentage points. Each subgroup must have at least 20 fitting and 20 evaluation examples, otherwise we mark the panel as “insufficient data”.

Mean topic gASCE $\downarrow$
Condition	MMLU	MMLU-Pro	TriviaQA	PopQA
Uncalibrated	0.106 $\pm$ 0.005	0.216 $\pm$ 0.008	0.115 $\pm$ 0.007	0.274 $\pm$ 0.009
Marginal HB	0.011 $\pm$ 0.001	0.027 $\pm$ 0.002	0.022 $\pm$ 0.002	0.041 $\pm$ 0.003
Topic-only	0.008 $\pm$ 0.001	0.015 $\pm$ 0.002	0.014 $\pm$ 0.002	0.009 $\pm$ 0.002
CoT-only	0.010 $\pm$ 0.001	0.018 $\pm$ 0.002	0.017 $\pm$ 0.002	0.032 $\pm$ 0.002
Joint	0.010 $\pm$ 0.001	0.014 $\pm$ 0.002	0.014 $\pm$ 0.002	0.012 $\pm$ 0.001

Appendix

Table 5: Mean and worst-topic calibration error (gASCE) on held-out examples across four datasets. Lower values indicate better topic-level calibration.

Figure 11: Topic-level reliability for all 39 MMLU topics, before calibration and after topic-only or CoT-only IGHB. Each point shows mean confidence and accuracy in a nonempty topic–confidence bin, using ten equal-width bins. Color identifies the topic and point size indicates sample count. The predictors and evaluation examples are the same as in Figure 6 .

Figure 12: CoT-group reliability on held-out MMLU responses, before calibration and after topic-only or CoT-only IGHB. This reverses the audit in Figure 11 : predictions are evaluated within CoT groups rather than topic groups. Each color denotes the Yes-group of one diagnostic question.

Figure 13: CoT subgroup errors after topic-only IGHB calibration on held-out MMLU responses from Qwen3-8B. Columns show all 39 embedding-derived topic groups, using the same names as Figure 9 . Rows show diagnostic questions, with Yes answers in the top panel and No answers in the bottom panel. The pooled topic error is repeated in the first row. Colors show mean predicted correctness minus observed correctness, in percentage points (pp); positive values indicate overprediction. Gray cells contain fewer than 20 responses.

Brier score $\downarrow$
Calibration	Labeler	MMLU	MMLU-Pro	TriviaQA	PopQA
Topic-only	—	0.127 $\pm$ 0.003	0.163 $\pm$ 0.004	0.185 $\pm$ 0.004	0.114 $\pm$ 0.004
CoT-only	Qwen3-4B	0.127 $\pm$ 0.003	0.163 $\pm$ 0.004	0.183 $\pm$ 0.004	0.129 $\pm$ 0.004
CoT-only	Qwen3-8B	0.123 $\pm$ 0.003	0.163 $\pm$ 0.004	0.174 $\pm$ 0.004	0.126 $\pm$ 0.004
CoT-only	Gemma4-31B	0.113 $\pm$ 0.003	0.141 $\pm$ 0.004	0.164 $\pm$ 0.004	0.125 $\pm$ 0.004
Joint	Qwen3-4B	0.124 $\pm$ 0.003	0.155 $\pm$ 0.004	0.178 $\pm$ 0.004	0.109 $\pm$ 0.004

Appendix

Table 6: Results across diagnostic labelers, audited over topics. Other baselines appear in Table 1 .

Brier score $\downarrow$
Calibration	MMLU	MMLU-Pro	TriviaQA	PopQA
Topic-only	0.127 $\pm$ 0.003	0.161 $\pm$ 0.004	0.181 $\pm$ 0.004	0.109 $\pm$ 0.004
CoT-only	0.112 $\pm$ 0.003	0.141 $\pm$ 0.004	0.159 $\pm$ 0.004	0.119 $\pm$ 0.004
Joint	0.112 $\pm$ 0.003	0.141 $\pm$ 0.004	0.158 $\pm$ 0.004	0.103 $\pm$ 0.004
Mean topic gASCE $\downarrow$
Topic-only	0.009 $\pm$ 0.001	0.015 $\pm$ 0.002	0.018 $\pm$ 0.002	0.009 $\pm$ 0.001

Appendix

Table 7: IGLB calibration on the held-out examples. Topic gASCE uses the same eligible topics and ten confidence bins as Table 5 . Lower is better.

Evaluated on MMLU
Calibrated on	Raw	HB	Topic-only	CoT-only	Joint
MMLU	0.2329	0.1366	0.1267	0.1127	0.1124
MMLU-Pro	0.2329	0.1418	0.1416	0.1177	0.1187
TriviaQA	0.2329	0.1950	0.1854	0.1284	0.1303
PopQA	0.2329	0.4385	0.4322	0.2399	0.2688
Evaluated on MMLU-Pro

Appendix

Table 8: Brier score under benchmark transfer. Rows identify the calibration benchmark; bold labels mark fitting on the target’s separate calibration split. Lower is better.

Evaluated on MMLU
Calibrated on	Raw	HB	Topic-only	CoT-only	Joint
MMLU	0.106	0.011	0.008	0.010	0.010
MMLU-Pro	0.106	0.017	0.023	0.017	0.017
TriviaQA	0.106	0.074	0.066	0.026	0.026
PopQA	0.106	0.322	0.316	0.131	0.158
Evaluated on MMLU-Pro

Appendix

Table 9: Membership-weighted mean gASCE over target-defined topics. Rows identify the calibration benchmark; bold labels mark fitting on the target’s separate calibration split. Lower is better.

Calibration	Brier	Mean topic gASCE	Worst-topic gASCE
Uncalibrated	0.166 $\pm$ 0.005	0.047 $\pm$ 0.003	0.552 $\pm$ 0.056
Topic-only	0.107 $\pm$ 0.003	0.011 $\pm$ 0.001	0.068 $\pm$ 0.017
CoT-only (10)	0.118 $\pm$ 0.003	0.019 $\pm$ 0.001	0.214 $\pm$ 0.038
Joint (10)	0.106 $\pm$ 0.003	0.010 $\pm$ 0.001	0.062 $\pm$ 0.017
CoT-only (15)	0.108 $\pm$ 0.003	0.016 $\pm$ 0.001	0.177 $\pm$ 0.037
Joint (15)	0.102 $\pm$ 0.003	0.010 $\pm$ 0.001	0.106 $\pm$ 0.023

Appendix

Table 10: Brier score and topic calibration for the safety prediction task, with ten or fifteen diagnostics, audited over topics. Values are estimates

\pm

one bootstrap standard error ( Section 5 ). Bold marks the lowest point estimate in each column.

CoT property	Diagnostic question	Yes
Request and response	Does the trace explicitly distinguish the user’s request from the response being evaluated when explaining its safety judgment?	4,399
Discussion and assistance	Does the trace explicitly distinguish discussing or describing a harmful activity from providing practical help to carry it out?	524
Harm mechanism	Does the trace describe a concrete way that the response could cause or facilitate harm? Do not count a bare harm-category name or the word unsafe.	1,250
Refusals and disclaimers	Does the trace assess whether a refusal, warning, or disclaimer changes the safety of the rest of the response? Do not count merely noting that a refusal, warning, or disclaimer is present.	183
Context dependence	Does the trace explicitly make its safety judgment depend on an assumption or condition about intent, authorization, audience, or use context? Do not count merely mentioning context.	352

Appendix

Table 11: Additional safety diagnostics questions. The “Yes” column reports the distribution of positive answers across the questions.

Language Model Calibration

Language model calibration studies whether models’ confidence estimates match their actual accuracy across tasks, prompts, and deployment settings.

See all 386 papers

Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
Apr 24, 2026Prakul Sunil Hiremath, Harshit R. HiremathConfidence Estimation in Language Models CoT Reasoning
Department of Computer Science and Engineering Visvesvaraya Technological University, Belagavi · Department of Computer Science and Business System SG Balekundri Institute of Technology, Belagavi
On Calibration of Large Language Models: From Response To Capability
Feb 14, 2026Sin-Han Yang, Cheng-Kuang Wu, Chieh-Yen Lin +3LLM Evaluation Confidence Estimation in Language Models
Appier AI Research · National Taiwan University
CALIBER: Calibrating Confidence Before and After Reasoning in Language Models
Jun 23, 2026Conor Finlay, Joshua Kurien, Saurabh Dash +2Confidence Estimation in Language Models Language Model Calibration
1Cohere Labs

Apr 24, 2026J/K move · Enter open · S save

Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models

Prakul Sunil Hiremath, Harshit R. Hiremath

Department of Computer Science and Engineering Visvesvaraya Technological University, Belagavi · Department of Computer Science and Business System SG Balekundri Institute of Technology, Belagavi

Confidence Estimation in Language Models CoT Reasoning Language Model Calibration LLM Reliability LLM Uncertainty Estimation LLM Reasoning

Open paper AskPDF arXiv

The ability of large language models (LLMs) to express calibrated uncertainty is important for safe deployment. Chain-of-thought (CoT) reasoning is widely used to improve accuracy and reliability, but its effect on calibration is not fully understood. We show that this picture is incomplete: in some settings, increasing the reasoning budget beyond a task-specific threshold can cause models to become systematically overconfident, assigning high confidence to incorrect answers. We call this phenomenon Calibration Drift Under Reasoning (CDUR) and study it both theoretically and empirically. We define reasoning budget B and analyze conditions under which Expected Calibration Error ECE(B) follows a non-monotonic pattern: it first decreases as reasoning corrects errors, then increases as longer reasoning produces internally consistent but incorrect explanations. We propose a Hypothesis Lock-In model based on autoregressive generation to explain this behavior. We evaluate Llama-3.1-8B and Llama-3.3-70B on 47 reasoning-trap questions across four reasoning budgets and three seeds (1,368 API calls; 574 valid responses). The 8B model shows non-monotonic calibration behavior, while results for the 70B model are limited to baseline evaluation and are inconclusive for budget-dependent effects. We introduce CABStop, a calibration-aware stopping rule that halts reasoning when confidence diverges from an auxiliary accuracy estimate. These results suggest that increasing reasoning depth does not always improve reliability and should be monitored carefully.

Abstract

Language Model Calibration

Explore similar work

Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models

On Calibration of Large Language Models: From Response To Capability

CALIBER: Calibrating Confidence Before and After Reasoning in Language Models