When Guessing is Rewarded: Rethinking Language Model Evaluation with Distributional Uncertainty Scoring

Thomas F Burns

cs.CLOct 5, 2025

When Guessing is Rewarded: Rethinking Language Model Evaluation with Distributional Uncertainty Scoring

Authors: Thomas F Burns

Organizations: Aleph Alpha Research

Abstract

Standard language model evaluation assigns scores to single predicted answers, rewarding high-confidence responses regardless of how residual probability mass is distributed over alternative options. This creates a systematic pressure toward overconfident guessing: under accuracy-based schemes, a model maximises its expected score by always committing to an answer rather than abstaining, even when its uncertainty is high. While penalty-based approaches partially address this by raising the confidence threshold for strategic guessing, they still treat all sub-threshold responses identically, ignoring a fundamental distinction in how models can express uncertainty - for example between hedging toward incorrect answers versus hedging toward "I don't know" responses. This paper introduces a novel evaluation metric to solve this problem of not considering a model's entire probability distribution over answer choices. The metric naturally distinguishes between harmful overconfidence in wrong answers and uncertainty expressed through abstention, providing scores in an interpretable default range. Through theoretical analysis and illustrative examples, the metric is shown to offer a more nuanced and aligned evaluation paradigm that incentivises models to express genuine uncertainty rather than guessing. Adapting 12 existing evaluation benchmarks to the metric's variants and measuring performance on six language models shows that for half of the tested benchmarks scores are negative across all tested models, indicating significant tendencies towards hallucination.

Figures & tables

Figure 1: Values of the optimal guessing threshold in DCS, as given by Proposition 1 for different values of the parameters $l_{c}$ and $l_{w}$ .

Figure 2: Mean MMLU scores as measured by accuracy, confidence-weighted accuracy, ternary score, and DCS across the evaluated models. All scores are multiplied by

100

for readability. Error bars represent the standard error (S.E.).

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	-24.5 $\pm$ 0.3	-15.1 $\pm$ 0.2	-12.1 $\pm$ 0.3	-10.6 $\pm$ 0.3	-13.9 $\pm$ 0.3	-19.6 $\pm$ 0.3
COPA	0.6 $\pm$ 0.4	2.9 $\pm$ 0.4	3.4 $\pm$ 0.4	10.2 $\pm$ 1.0	3.0 $\pm$ 0.3	3.2 $\pm$ 1.1
GPQA	-21.6 $\pm$ 0.4	-44.6 $\pm$ 0.7	-43.0 $\pm$ 1.1	-44.7 $\pm$ 1.6	-41.9 $\pm$ 1.0	-39.3 $\pm$ 0.5
HellaSwag	-34.0 $\pm$ 0.2	-26.2 $\pm$ 0.1	-23.2 $\pm$ 0.2	-19.0 $\pm$ 0.2	-27.8 $\pm$ 0.2	-37.2 $\pm$ 0.3
MMLU	-33.8 $\pm$ 0.5	0.8 $\pm$ 0.5	14.0 $\pm$ 2.1	18.3 $\pm$ 2.6	19.0 $\pm$ 1.9	9.3 $\pm$ 1.6
MMLU Pro	-61.5 $\pm$ 0.3	-56.8 $\pm$ 0.3	-38.8 $\pm$ 1.9	-34.6 $\pm$ 2.6	-38.9 $\pm$ 1.8	-40.3 $\pm$ 1.4

Table 1: Mean DCS (

\pm

S.E.) across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) DCS per benchmark is shown in bold and the second best is underlined .

Benchmark	$\Delta$ Accuracy	$\Delta$ C-weighted acc.	$\Delta$ Ternary
ARC	-28.9	-19.7	-23.8
COPA	-14.8	-4.6	-8.6
GPQA	-66.5	-49.6	-7.4
HellaSwag	-78.8	-41.4	-49.6
MMLU	-51.9	-37.8	-12.2
MMLU Pro	-77.6	-63.9	-14.6

Table 2: Per-benchmark mean gap between DCS and each comparison metric (DCS minus the metric), averaged across the six models. As elsewhere, quantities are multiplied by 100 fo readability. Negative values indicate DCS scores below the comparison metric.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Asset	License (summary)
transformers library	Apache License 2.0 (repository LICENSE )
eval-framework	Apache License 2.0 (repository LICENSE )
Model checkpoints (Hugging Face model cards)
microsoft/DialoGPT-medium	MIT
meta-llama/Llama-3.2-3B-Instruct	Llama 3.2 Community License Agreement
meta-llama/Llama-3.1-8B-Instruct	Llama 3.1 Community License Agreement

Appendix

Table 3: Primary licenses for major third-party assets used in this work (see also Appendix A.4.1 for model-card URLs). Where a benchmark is distributed under several compatible terms depending on mirror, the license cited is that stated on the Hugging Face dataset card or canonical repository at access time.

Desired threshold $p_{c}^{*}$	Loadings ( $l_{c}=1,\ l_{w}$ )
$0.10$	$\displaystyle l_{w}=1/9$
$0.20$	$\displaystyle l_{w}=1/4$
$0.30$	$\displaystyle l_{w}=3/7$
$0.40$	$\displaystyle l_{w}=2/3$
$0.50$	$\displaystyle l_{w}=1$
$0.60$	$\displaystyle l_{w}=3/2$

Appendix

Table 4: Example loadings for various desired optimal guessing thresholds

p_{c}^{*}

. These loadings use the convention of a fixed

l_{c}=1

and sets

l_{w}=p_{c}^{*}/(1-p_{c}^{*})

Accuracy	if $\max_{a}a\in\mathcal{P}_{\mathcal{A}}$ is $c$ , $s=1$ , else $s=0$
C-weighted acc.	if $\max_{a}a\in\mathcal{P}_{\mathcal{A}}$ is $p$ , $s=p_{c}$ , else $s=0$
Ternary score	let $g=\max_{a}a\in\mathcal{P}_{\mathcal{A}}$ ; if $g=c$ , $s=1$ ; if $g=\textsc{idk}$ , $s=0$ ; otherwise $s=-1$

Appendix

Table 5: Computed comparison metrics. Here the set of answer probabilities is denoted

\mathcal{P}_{\mathcal{A}}

, the correct answer

c

, and the IDK response idk , with the probabilities notation as in Definition 1 . ‘C-weighted acc.’ is an abbreviation of ‘confidence-weighted accuracy’.

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	7.3 $\pm$ 0.8	10.0 $\pm$ 0.5	4.1 $\pm$ 0.6	16.6 $\pm$ 1.2	14.0 $\pm$ 1.1	25.4 $\pm$ 1.4
COPA	4.0 $\pm$ 2.0	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	53.0 $\pm$ 5.0	0.0 $\pm$ 0.0	55.0 $\pm$ 5.0
GPQA	4.6 $\pm$ 0.9	32.1 $\pm$ 2.0	28.6 $\pm$ 1.9	30.3 $\pm$ 2.0	32.1 $\pm$ 2.0	36.5 $\pm$ 2.1
HellaSwag	28.8 $\pm$ 1.4	48.4 $\pm$ 0.5	62.8 $\pm$ 1.5	34.7 $\pm$ 1.5	69.0 $\pm$ 1.5	61.8 $\pm$ 1.5
MMLU	18.6 $\pm$ 1.2	62.1 $\pm$ 0.4	61.1 $\pm$ 1.4	59.7 $\pm$ 1.5	67.8 $\pm$ 1.4	69.8 $\pm$ 1.4
MMLU Pro	7.8 $\pm$ 0.8	31.3 $\pm$ 0.4	37.2 $\pm$ 1.5	34.3 $\pm$ 1.6	40.5 $\pm$ 1.5	43.8 $\pm$ 1.5

Appendix

Table 6: Mean accuracy (

\pm

S.E.) across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) score per benchmark is shown in bold and the second best is underlined .

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	1.7 $\pm$ 0.2	3.0 $\pm$ 0.2	1.2 $\pm$ 0.2	5.5 $\pm$ 0.4	4.3 $\pm$ 0.3	6.7 $\pm$ 0.4
COPA	1.5 $\pm$ 0.7	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	24.8 $\pm$ 2.4	0.0 $\pm$ 0.0	24.8 $\pm$ 2.3
GPQA	1.2 $\pm$ 0.2	10.9 $\pm$ 0.7	11.6 $\pm$ 0.8	15.0 $\pm$ 1.1	12.4 $\pm$ 0.8	11.7 $\pm$ 0.7
HellaSwag	7.3 $\pm$ 0.4	11.9 $\pm$ 0.1	16.4 $\pm$ 0.4	9.6 $\pm$ 0.4	17.8 $\pm$ 0.4	18.2 $\pm$ 0.5
MMLU	6.9 $\pm$ 0.4	42.0 $\pm$ 0.3	49.8 $\pm$ 1.3	55.7 $\pm$ 1.4	52.6 $\pm$ 1.2	47.3 $\pm$ 1.0
MMLU Pro	1.4 $\pm$ 0.2	13.0 $\pm$ 0.2	23.9 $\pm$ 1.1	29.0 $\pm$ 1.4	23.4 $\pm$ 1.0	21.6 $\pm$ 0.9

Appendix

Table 7: Mean confidence-weighted accuracy (

\pm

S.E.) across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) score per benchmark is shown in bold and the second best is underlined .

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	-6.9 $\pm$ 1.5	8.2 $\pm$ 0.6	3.7 $\pm$ 0.7	14.0 $\pm$ 1.3	11.9 $\pm$ 1.2	15.9 $\pm$ 1.8
COPA	3.0 $\pm$ 2.2	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	48.0 $\pm$ 5.9	0.0 $\pm$ 0.0	24.0 $\pm$ 9.0
GPQA	-10.1 $\pm$ 1.8	-35.8 $\pm$ 4.0	-42.8 $\pm$ 3.9	-39.4 $\pm$ 3.9	-35.8 $\pm$ 4.0	-27.0 $\pm$ 4.1
HellaSwag	-40.3 $\pm$ 2.9	27.9 $\pm$ 0.8	49.5 $\pm$ 2.3	28.6 $\pm$ 1.8	40.6 $\pm$ 2.8	23.6 $\pm$ 3.1
MMLU	-40.7 $\pm$ 2.3	24.3 $\pm$ 0.8	22.3 $\pm$ 2.9	19.6 $\pm$ 2.9	35.6 $\pm$ 2.8	39.7 $\pm$ 2.7
MMLU Pro	-58.2 $\pm$ 2.0	-37.3 $\pm$ 0.8	-25.5 $\pm$ 3.0	-31.4 $\pm$ 3.2	-19.0 $\pm$ 3.0	-11.9 $\pm$ 3.1

Appendix

Table 8: Mean ternary score (

\pm

S.E.) across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) score per benchmark is shown in bold and the second best is underlined .

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	-19.8 $\pm$ 0.3	-12.3 $\pm$ 0.2	-9.9 $\pm$ 0.2	-8.7 $\pm$ 0.3	-11.3 $\pm$ 0.3	-15.9 $\pm$ 0.3
COPA	0.5 $\pm$ 0.3	2.3 $\pm$ 0.3	2.7 $\pm$ 0.3	8.3 $\pm$ 0.8	2.4 $\pm$ 0.3	2.6 $\pm$ 0.9
GPQA	-17.5 $\pm$ 0.4	-36.2 $\pm$ 0.6	-34.9 $\pm$ 0.9	-36.3 $\pm$ 1.3	-34.0 $\pm$ 0.8	-31.9 $\pm$ 0.4
HellaSwag	-27.6 $\pm$ 0.2	-21.3 $\pm$ 0.1	-18.8 $\pm$ 0.2	-15.4 $\pm$ 0.2	-22.6 $\pm$ 0.2	-30.2 $\pm$ 0.2
MMLU	-27.4 $\pm$ 0.4	0.7 $\pm$ 0.4	11.4 $\pm$ 1.7	14.9 $\pm$ 2.1	15.4 $\pm$ 1.5	7.6 $\pm$ 1.3
MMLU Pro	-49.9 $\pm$ 0.3	-46.1 $\pm$ 0.3	-31.5 $\pm$ 1.5	-28.1 $\pm$ 2.1	-31.6 $\pm$ 1.5	-32.7 $\pm$ 1.1

Appendix

Table 9: Mean DCS (

\pm

S.E.) using summed probabilities over 10 abstention phrase variations across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) DCS per benchmark is shown in bold and the second best is underlined .

Explore similar work

Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations
Jan 12, 2026Yuxi Xia, Dennis Ulmer, Terra Blevins +3Confidence Estimation
Faculty of Computer Science, UniVie Doctoral School Computer Science · Faculty of Philological and Cultural Studies, University of Vienna, Austria · ILLC, University of Amsterdam, Netherlands +2
Position: Evaluation Scores Are Perishable Knowledge Claims
Jul 28, 2026Sankalp Gilda, Shlok GildaConfidence Calibration Position
DeepThought Solutions · Meta · University of Florida
Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs
Sep 17, 2025Edward Phillips, Sean Wu, Soheila Molaei +3Large Language Model Uncertainty Large Language Model Hallucination
Department of Engineering Science, University of Oxford · GlaxoSmithKline · Oxford Suzhou Centre for Advanced Research

Jan 12, 2026, cs.CLJ/K move · Enter open · S save

Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations

Yuxi Xia, Dennis Ulmer, Terra Blevins, Yihong Liu+2

Faculty of Computer Science, UniVie Doctoral School Computer Science · Faculty of Philological and Cultural Studies, University of Vienna, Austria · ILLC, University of Amsterdam, Netherlands+2

Confidence Estimation

Open paper AskPDF arXiv

Confidence estimation (CE) indicates how reliable the answers of large language models are and impacts user trust and decision-making. Existing evaluations mainly concern the alignment between confidence and correctness, but ignore the variability of language: confidence estimates should remain consistent under semantically equivalent prompts or answer variations, while changing when answer meaning differs, as this may indicate a change in correctness. Therefore, we introduce a novel evaluation framework based on three complementary properties: \textbf{robustness} to prompt perturbations, \textbf{stability} across semantically equivalent answers, and \textbf{sensitivity} to semantically different answers. We show that these metrics are largely independent from existing CE metrics, and that common CE methods often fail on them: while most methods achieve high robustness and stability, they struggle to distinguish semantically different answers, potentially because they do not effectively leverage generation-side information. Overall, our framework exposes overlooked limitations of current CE evaluations and provides guidance for selecting confidence estimators for real-world applications.

Abstract

Explore similar work

Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations

Position: Evaluation Scores Are Perishable Knowledge Claims

Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs