Each Insufficient in Its Own Way: Calibrating Language Models Across Varying Evidence States

Jiayi Li; Dennis L. Shung; Bradly C. Stadie

cs.CLOct 23, 2026

Each Insufficient in Its Own Way: Calibrating Language Models Across Varying Evidence States

Authors: Jiayi Li, Dennis L. Shung, Bradly C. Stadie

Organizations: Northwestern University · Mayo Clinic

Abstract

How can a language model know whether it has enough evidence to answer? Language models tend to answer even when the evidence they have is insufficient. Deciding whether to answer or gather more evidence requires a calibrated estimate of how likely an answer is to be correct with the available evidence. However, most calibration work treats the input as fixed, without asking how this estimate should change with the evidence. We introduce evidence-state calibration, which requires a model's confidence before answering to match its probability of answering correctly and to change accordingly whenever the evidence changes this probability. We propose \ours{} (Sufficiency Confidence Estimation), which fine-tunes the confidence already present in a model's next-token distribution to match how often the model answers correctly from each evidence state. Across clinical diagnosis and multi-hop question answering with four models from two families, \ours{} reduces the Brier score by up to 60% and increases AUROC by up to 0.16. Its confidence matches the empirical success rate and responds not only to the amount of evidence but also to which evidence is present, showing that models can learn to judge whether their current evidence is enough to answer.

Figures & tables

Figure 1: Given a question with two different sets of evidence, the model’s probability of answering correctly (gray) differs, and its confidence before answering (yellow) should match it for each. (a) Calibrated confidence matches both. (b) Miscalibrated confidence mismatches the dynamic of the true probability.

Figure 2: Overview of SuffiCE. Each evidence state $x=(q,I)$ pairs an input question $q$ with the available evidence $I$ . Top: A frozen answering policy samples $K$ answers per evidence state, and the fraction graded correct is the empirical success rate $\hat{p}(x)$ . Bottom: The estimator reads its confidence from the softmax over the logits of A (Answer) and S (Seek), and is trained with soft-label binary cross-entropy to match $\hat{p}(x)$ .

Kimi K2.6
	MIMIC			MedQA			MuSiQue
Method	Brier $\downarrow$	AUROC $\uparrow$	$\rho_{\Delta}$ $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	$\rho_{\Delta}$ $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	$\rho_{\Delta}$ $\uparrow$
Base logit	0.219 $\pm$ 0.000	0.686 $\pm$ 0.000	0.320 $\pm$ 0.000	0.223 $\pm$ 0.000	0.707 $\pm$ 0.000	0.413 $\pm$ 0.000	0.184 $\pm$ 0.000	0.750 $\pm$ 0.000	0.430 $\pm$ 0.000
Verbalized	0.131 $\pm$ 0.001	0.686 $\pm$ 0.003	0.247 $\pm$ 0.013	0.162 $\pm$ 0.001	0.650 $\pm$ 0.001	0.374 $\pm$ 0.010	0.174 $\pm$ 0.003	0.724 $\pm$ 0.003	0.412 $\pm$ 0.011
Platt	0.124 $\pm$ 0.000	0.686 $\pm$ 0.000	0.296 $\pm$ 0.000	0.150 $\pm$ 0.000	0.707 $\pm$ 0.000	0.438 $\pm$ 0.000	0.142 $\pm$ 0.000	0.750 $\pm$ 0.000	0.431 $\pm$ 0.000
SuffiCE	0.105 $\pm$ 0.002	0.844 $\pm$ 0.005	0.474 $\pm$ 0.045	0.141 $\pm$ 0.005	0.795 $\pm$ 0.033	0.467 $\pm$ 0.064	0.134 $\pm$ 0.004	0.787 $\pm$ 0.005	0.632 $\pm$ 0.009

Table 1: Results on the three datasets for Kimi K2.6, Qwen3.8-27B, Qwen3.5-9B, and Qwen3.5-4B, comparing our method with baselines. Results are mean

\pm

SD over three seeds. The best result for each model is in bold.

Figure 3: Evidence states of MuSiQue for chain and branching questions. Hatched cells give the number of distractors padding each state to three paragraphs.

Figure 4: SuffiCE brings confidence closer to perfect calibration on every dataset. Calibration curves on the test set (a) before and (b) after SuffiCE.

	Answered	Base	Verb.	SuffiCE
MIMIC	10%	97.2	95.3	100.0
	30%	93.4	95.3	98.5
	50%	89.9	92.4	95.5
MedQA	10%	93.7	74.3	99.1
	30%	77.8	73.4	84.2
	50%	65.4	63.3	74.0

Table 2: Accuracy (%) when the model answers the given percentage of questions it is most confident about.

Figure 5: Training across evidence states improves both calibration and evidence sensitivity. Estimators trained for the same number of epochs on full states only versus on all evidence states (SuffiCE).

Figure 6: SuffiCE tracks the empirical success rate as supporting paragraphs are added, while the base logit rises with every paragraph. MuSiQue test set, for (a) chain and (b) branching questions.

Figure 7: With the same amount of evidence, SuffiCE tells which states can be answered. Each MuSiQue state shows two of the three supporting paragraphs, for (a) chain and (b) branching questions. Only SuffiCE stays low until the final hop

e_{3}

is shown.

Panel of Table 3 — Table 3: A MedQA case of Hirschsprung disease (Qwen3.5-9B). Each state is the set of evidence shown. Success is the fraction of ten sampled answers that are correct. Base and SuffiCE give the sufficiency confidence.

Figure 8: Qwen estimators of three sizes are trained to predict the success rates of Kimi K2.6, compared with Kimi K2.6 predicting its own (self).

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Mean success rate $\hat{p}$
	MIMIC	MedQA	MuSiQue
Answer	4-way choice	free text	short answer
Problems	140 / 60	146 / 63	200 / 80
States	840 / 360	728 / 302	1,500 / 600
Qwen3.5-4B	0.82 / 0.81	0.45 / 0.44	0.27 / 0.25
Qwen3.5-9B	0.84 / 0.81	0.50 / 0.50	0.29 / 0.25

Appendix

Table 4: Dataset statistics, given as train / test. The mean success rate

\hat{p}

of each answering model is taken over all evidence states.

Panel of Table 5 — Table 5: The two MuSiQue question types, with sub-questions from MuSiQue’s decompositions (capitalization normalized). # $k$ stands for the answer of hop $k$ , supported by paragraph $e_{k}$ . In the chain, each hop depends on the previous one, so each stage has its own color. In the branching question, $e_{1}$ and $e_{2}$ are independent and share a color, and $e_{3}$ combines their answers.

Dataset	Noise floor	SuffiCE Brier
MIMIC	0.003	0.092
MedQA	0.008	0.111
MuSiQue	0.007	0.097

Appendix

Table 6: Noise floor, the expected Brier score of a perfect estimator against ten-answer success rates, compared with the Brier score of SuffiCE (Table 1 ). Both are averaged over the four answering models, on the test set.

Success-rate estimation
Setting	Value
Answers per state $K$	10
Temperature	1.0
Top- $p$ / top- $k$	none
LLM judge (if used)	GPT-5.6 Luna
Estimator training

Appendix

Table 7: Settings shared by all models and datasets.

Dataset	{task line}
MIMIC	Which of the following is the most likely diagnosis: appendicitis, cholecystitis, diverticulitis, or pancreatitis?
MedQA	What is the most likely diagnosis?
MuSiQue	(none, the question is the whole task)

Appendix

Table 8: The dataset-specific task line of the sufficiency and verbalized prompts.

Figure 9: Calibration curves of Qwen3.5-4B, Qwen3.5-9B, Qwen3.8-27B, and Kimi K2.6 on the test set, (a) before and (b) after SuffiCE.

Figure 10: Selective answering on the test set of every dataset (rows) and model (columns). Each curve plots the accuracy of the answered questions against the percentage answered, answering the questions with the highest confidence first. Higher is better. Platt scaling preserves the ranking of the base logit, so their curves coincide.

MIMIC
Method	Brier $\downarrow$	AUROC $\uparrow$	$\rho_{\Delta}$ $\uparrow$
Full only	0.130 $\pm$ 0.005	0.745 $\pm$ 0.014	0.251 $\pm$ 0.024
SuffiCE	0.098 $\pm$ 0.010	0.819 $\pm$ 0.028	0.393 $\pm$ 0.034
MedQA
Full only	0.131 $\pm$ 0.004	0.782 $\pm$ 0.007	0.554 $\pm$ 0.008
SuffiCE	0.107 $\pm$ 0.005	0.815 $\pm$ 0.011	0.630 $\pm$ 0.012

Appendix

Table 9: Step-matched ablation for Qwen3.5-9B. The full-only estimator is trained for the same number of steps as SuffiCE.

Kimi K2.6
	MIMIC		MedQA		MuSiQue
Method	$\Delta$ MAE $\downarrow$	$\rho_{\text{within}}$ $\uparrow$	$\Delta$ MAE $\downarrow$	$\rho_{\text{within}}$ $\uparrow$	$\Delta$ MAE $\downarrow$	$\rho_{\text{within}}$ $\uparrow$
Base logit	0.333 $\pm$ 0.000	0.217 $\pm$ 0.000	0.282 $\pm$ 0.000	0.290 $\pm$ 0.000	0.355 $\pm$ 0.000	0.164 $\pm$ 0.000
Verbalized	0.182 $\pm$ 0.001	0.222 $\pm$ 0.007	0.193 $\pm$ 0.002	0.247 $\pm$ 0.004	0.367 $\pm$ 0.003	0.123 $\pm$ 0.029
Platt	0.190 $\pm$ 0.000	0.210 $\pm$ 0.000	0.202 $\pm$ 0.000	0.290 $\pm$ 0.000	0.351 $\pm$ 0.000	0.173 $\pm$ 0.000
SuffiCE	0.188 $\pm$ 0.008	0.496 $\pm$ 0.022	0.188 $\pm$ 0.013	0.430 $\pm$ 0.059	0.292 $\pm$ 0.003	0.216 $\pm$ 0.021

Appendix

Table 10: Evidence-state analysis for every answering model.

\Delta

MAE compares changes in confidence with changes in empirical success between states of the same problem, and

\rho_{\text{within}}

correlates confidence with success among problems that share a state type. The best result for each model is in bold.

Train MIMIC $\rightarrow$ test MedQA
Metric	Base logit	Cross-domain	In-domain
Brier $\downarrow$	0.268 $\pm$ 0.000	0.189 $\pm$ 0.027	0.107 $\pm$ 0.005
AUROC $\uparrow$	0.746 $\pm$ 0.000	0.779 $\pm$ 0.009	0.815 $\pm$ 0.011
$\rho_{\Delta}$ $\uparrow$	0.367 $\pm$ 0.001	0.401 $\pm$ 0.027	0.630 $\pm$ 0.012
$\rho_{\text{within}}$ $\uparrow$	0.398 $\pm$ 0.000	0.491 $\pm$ 0.029	0.592 $\pm$ 0.024
Train MedQA $\rightarrow$ test MIMIC

Appendix

Table 11: Cross-domain transfer between the clinical datasets (Qwen3.5-9B). Cross-domain and in-domain are SuffiCE trained on the other dataset and on the test dataset itself. The best result in each row is in bold and the second best in italics.

Panel of Table 13 — Table 13: Case study on a 3hop2 MuSiQue question (Qwen3.5-9B), where the branches $e_{1}$ and $e_{2}$ together lead to the final hop $e_{3}$ . Columns are evidence states, marked by which supporting facts are shown ( $\varnothing$ for none).

Panel of Table 14 — Table 14: A second case study on a 3hop2 MuSiQue question (Qwen3.5-9B), in the format of Table 13 . The shaded state has both branches but not the final hop.

Objective	Assess and diagnose the infant brought in by a concerned mother due to bilious vomiting and inability to pass meconium, with observed abdominal distension. Answer: Hirschsprung’s disease
1 History	Demographics: 2-day-old male infant. History: The infant was born via a home birth without complications. The mother reports that the infant has had bilious vomiting and has not been able to pass meconium since birth. She also notes that the infant’s abdomen appears distended. Primary symptom: Bilious vomiting. Secondary symptoms: Inability to pass meconium, Abdominal distension. Past medical history: Newborn, no prior medical history. Birth was uneventful. Social history: The infant is the first child of the family. No known family history of genetic or congenital diseases. Review of systems: The infant shows signs of distress, possibly due to abdominal discomfort.
2 PE	Vital signs: Temperature 37.5 ∘ C (99.5 ∘ F), Blood pressure not applicable for age, Heart rate 145 bpm, Respiratory rate 40 breaths/min. Abdominal examination: Inspection: Noticeable distension. Auscultation: Decreased bowel sounds. Percussion: Tympanic sound noted, indicating gas. Palpation: The abdomen is firm and distended, with palpable bowel loops; the infant shows signs of discomfort during palpation.
3 Biopsy	Rectal biopsy, histopathology: Absence of ganglion cells in the submucosa and myenteric plexus, consistent with Hirschsprung’s disease.
4 Imaging	Abdominal X-ray, findings: Dilated loops of bowel with no gas in the rectum, suggestive of a distal obstruction.

Appendix

Table 15: Full input of the MedQA case of Table 3 and the evidence states shown there. Each state shows the objective followed by the listed sections, verbatim.

Panel of Table 16 — Table 16: Full input and all eight evidence states of the 3hop2 MuSiQue question of Table 13 . Supporting paragraphs $e_{1}$ – $e_{3}$ and distractors $d_{1}$ – $d_{3}$ are shown verbatim. The six paragraphs of each question are shuffled once into a fixed order, here $e_{2},d_{1},d_{2},e_{1},e_{3},d_{3}$ , and every state shows its paragraphs in that order, so a paragraph’s position does not reveal the state. Every state except the question-only one shows three paragraphs, padded with distractors.

Confidence Estimation in Language Models

Confidence estimation in language models measures whether a model’s uncertainty signals predict answer correctness and support reliable decisions.

See all 248 papers

Jun 23, 2026

CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success. Existing methods typically elicit confidence only once: either before thinking or after answering. We argue that confidence in reasoning models is state-dependent: before thinking, confidence should estimate the chance of the model correctly solving the prompt, while after thinking it should predict whether the realized answer is likely to be correct. This distinction determines the appropriate supervision target: prompt-level success should supervise confidence estimates made after seeing the prompt, while individual answer-level correctness should supervise confidence estimates made after answering. We introduce CALIBER (Calibration Before and After Reasoning), which elicits both estimates and supervises each with the target matched to its information state. Under this unified protocol, CALIBER reduces Expected Calibration Error (ECE) by 52.5% over the strongest single-confidence baseline on BigMathDigits for the 7B model, while achieving the best Brier score and AUROC, and remains within 2.1 points of the best accuracy. Further, on a larger 30B model, CALIBER achieves the best ECE on BigMathDigits while remaining competitive in Brier score and AUROC. Out of distribution, it achieves the best ECE and Brier score on GPQA and TriviaQA, and remains competitive on SimpleQA. Ablations further show that this position-target alignment is most beneficial under distribution shift where it consistently reduces calibration error across all out-of-distribution benchmarks.

Conor Finlay, Joshua Kurien, Saurabh Dash +2

1Cohere Labs

Confidence Estimation in Language Models Language Model Calibration LLM Uncertainty Estimation Large Reasoning Models Confidence Estimation Verbalized Confidence Estimation

Ask PDF

Jan 26, 2026

Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration

Knowledge probing quantifies how much relational knowledge a language model (LM) has acquired during pre-training. Existing knowledge probes evaluate model capabilities through metrics like prediction accuracy and precision. Such evaluations fail to account for the model's reliability, reflected in the calibration of its confidence scores. In this paper, we establish an empirical framework to evaluate the calibration of relational knowledge in a closed answer-set setting. We cover three modalities of model confidence: (1) intrinsic confidence, (2) paraphrase consistency and (3) linguistic confidence. Our extensive analysis of 21 causal and 8 masked language models reveals that most models, but especially those pre-trained with the masking objective, are overconfident. The best-calibrated scores come from confidence estimates that account for inconsistencies due to statement rephrasing. Moreover, calibration improves when increasing the number of answer options.

Christopher Kissling, Elena Merdjanovska, Alan Akbik

Humboldt-Universität zu Berlin · Science of Intelligence

Language Model Probing Language Model Calibration Factual Knowledge in Language Models Confidence Estimation in Language Models

Ask PDF

Sep 27, 2026

Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models

Reinforcement learning with binary correctness rewards trains correctness, not calibrated confidence. The confidence that reasoning models verbalize is systematically overconfident, and the problem is not merely one of scale: verbalized confidence tracks how willing a model is to commit to an answer, not how likely the answer is to be right. Post-hoc rescaling therefore fits one distribution but rarely transfers. We look inside the model instead. On factual question answering, a linear probe on the hidden state between the chain of thought and the answer is substantially better calibrated: its expected calibration error is 5 to 38 times lower than that of the verbalized score across four benchmarks and two model families. However, when used to pick among N sampled answers, that same probe nearly ties majority voting yet falls far short of the oracle. Internal states answer "how certain am I" well and "which answer is right" poorly, so the signal should be reported as a confidence rather than used to select answers. As a result, we introduce probe-guided self-distillation (Probe-SD): score a model's own sampled traces with the probe, overwrite the confidence each trace states, and finetune the base checkpoint of the same family, so nothing but the model itself remains at test time. On Qwen3-14B, Probe-SD cuts ECE from 0.178 to 0.024 in-domain and from 0.542 to 0.113 out-of-domain, where it also beats post-hoc recalibration and self-consistency distillation. The resulting confidence is well-calibrated and useful for weighted voting, behaviors previously attributed to online RL, here obtained with supervised finetuning alone.

Yadong Xi, Rongsheng Zhang, Tangjie Lv +2

NetEase · Amazon · Singapore University of Technology and Design

Confidence Estimation in Language Models Language Model Calibration Language Model Distillation LLM Uncertainty Estimation

Ask PDF

Abstract

Confidence Estimation in Language Models

Explore similar work

CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration

Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models