When Guessing is Rewarded: Rethinking Language Model Evaluation with Distributional Uncertainty Scoring

Thomas F Burns

cs.CLOct 5, 2025

When Guessing is Rewarded: Rethinking Language Model Evaluation with Distributional Uncertainty Scoring

Authors: Thomas F Burns

Organizations: Aleph Alpha Research

Abstract

Standard language model evaluation assigns scores to single predicted answers, rewarding high-confidence responses regardless of how residual probability mass is distributed over alternative options. This creates a systematic pressure toward overconfident guessing: under accuracy-based schemes, a model maximises its expected score by always committing to an answer rather than abstaining, even when its uncertainty is high. While penalty-based approaches partially address this by raising the confidence threshold for strategic guessing, they still treat all sub-threshold responses identically, ignoring a fundamental distinction in how models can express uncertainty - for example between hedging toward incorrect answers versus hedging toward "I don't know" responses. This paper introduces a novel evaluation metric to solve this problem of not considering a model's entire probability distribution over answer choices. The metric naturally distinguishes between harmful overconfidence in wrong answers and uncertainty expressed through abstention, providing scores in an interpretable default range. Through theoretical analysis and illustrative examples, the metric is shown to offer a more nuanced and aligned evaluation paradigm that incentivises models to express genuine uncertainty rather than guessing. Adapting 12 existing evaluation benchmarks to the metric's variants and measuring performance on six language models shows that for half of the tested benchmarks scores are negative across all tested models, indicating significant tendencies towards hallucination.

Figures & tables

Figure 1: Values of the optimal guessing threshold in DCS, as given by Proposition 1 for different values of the parameters $l_{c}$ and $l_{w}$ .

Figure 2: Mean MMLU scores as measured by accuracy, confidence-weighted accuracy, ternary score, and DCS across the evaluated models. All scores are multiplied by

100

for readability. Error bars represent the standard error (S.E.).

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	-24.5 $\pm$ 0.3	-15.1 $\pm$ 0.2	-12.1 $\pm$ 0.3	-10.6 $\pm$ 0.3	-13.9 $\pm$ 0.3	-19.6 $\pm$ 0.3
COPA	0.6 $\pm$ 0.4	2.9 $\pm$ 0.4	3.4 $\pm$ 0.4	10.2 $\pm$ 1.0	3.0 $\pm$ 0.3	3.2 $\pm$ 1.1
GPQA	-21.6 $\pm$ 0.4	-44.6 $\pm$ 0.7	-43.0 $\pm$ 1.1	-44.7 $\pm$ 1.6	-41.9 $\pm$ 1.0	-39.3 $\pm$ 0.5
HellaSwag	-34.0 $\pm$ 0.2	-26.2 $\pm$ 0.1	-23.2 $\pm$ 0.2	-19.0 $\pm$ 0.2	-27.8 $\pm$ 0.2	-37.2 $\pm$ 0.3
MMLU	-33.8 $\pm$ 0.5	0.8 $\pm$ 0.5	14.0 $\pm$ 2.1	18.3 $\pm$ 2.6	19.0 $\pm$ 1.9	9.3 $\pm$ 1.6
MMLU Pro	-61.5 $\pm$ 0.3	-56.8 $\pm$ 0.3	-38.8 $\pm$ 1.9	-34.6 $\pm$ 2.6	-38.9 $\pm$ 1.8	-40.3 $\pm$ 1.4

Table 1: Mean DCS (

\pm

S.E.) across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) DCS per benchmark is shown in bold and the second best is underlined .

Benchmark	$\Delta$ Accuracy	$\Delta$ C-weighted acc.	$\Delta$ Ternary
ARC	-28.9	-19.7	-23.8
COPA	-14.8	-4.6	-8.6
GPQA	-66.5	-49.6	-7.4
HellaSwag	-78.8	-41.4	-49.6
MMLU	-51.9	-37.8	-12.2
MMLU Pro	-77.6	-63.9	-14.6

Table 2: Per-benchmark mean gap between DCS and each comparison metric (DCS minus the metric), averaged across the six models. As elsewhere, quantities are multiplied by 100 fo readability. Negative values indicate DCS scores below the comparison metric.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Asset	License (summary)
transformers library	Apache License 2.0 (repository LICENSE )
eval-framework	Apache License 2.0 (repository LICENSE )
Model checkpoints (Hugging Face model cards)
microsoft/DialoGPT-medium	MIT
meta-llama/Llama-3.2-3B-Instruct	Llama 3.2 Community License Agreement
meta-llama/Llama-3.1-8B-Instruct	Llama 3.1 Community License Agreement

Appendix

Table 3: Primary licenses for major third-party assets used in this work (see also Appendix A.4.1 for model-card URLs). Where a benchmark is distributed under several compatible terms depending on mirror, the license cited is that stated on the Hugging Face dataset card or canonical repository at access time.

Desired threshold $p_{c}^{*}$	Loadings ( $l_{c}=1,\ l_{w}$ )
$0.10$	$\displaystyle l_{w}=1/9$
$0.20$	$\displaystyle l_{w}=1/4$
$0.30$	$\displaystyle l_{w}=3/7$
$0.40$	$\displaystyle l_{w}=2/3$
$0.50$	$\displaystyle l_{w}=1$
$0.60$	$\displaystyle l_{w}=3/2$

Appendix

Table 4: Example loadings for various desired optimal guessing thresholds

p_{c}^{*}

. These loadings use the convention of a fixed

l_{c}=1

and sets

l_{w}=p_{c}^{*}/(1-p_{c}^{*})

Accuracy	if $\max_{a}a\in\mathcal{P}_{\mathcal{A}}$ is $c$ , $s=1$ , else $s=0$
C-weighted acc.	if $\max_{a}a\in\mathcal{P}_{\mathcal{A}}$ is $p$ , $s=p_{c}$ , else $s=0$
Ternary score	let $g=\max_{a}a\in\mathcal{P}_{\mathcal{A}}$ ; if $g=c$ , $s=1$ ; if $g=\textsc{idk}$ , $s=0$ ; otherwise $s=-1$

Appendix

Table 5: Computed comparison metrics. Here the set of answer probabilities is denoted

\mathcal{P}_{\mathcal{A}}

, the correct answer

c

, and the IDK response idk , with the probabilities notation as in Definition 1 . ‘C-weighted acc.’ is an abbreviation of ‘confidence-weighted accuracy’.

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	7.3 $\pm$ 0.8	10.0 $\pm$ 0.5	4.1 $\pm$ 0.6	16.6 $\pm$ 1.2	14.0 $\pm$ 1.1	25.4 $\pm$ 1.4
COPA	4.0 $\pm$ 2.0	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	53.0 $\pm$ 5.0	0.0 $\pm$ 0.0	55.0 $\pm$ 5.0
GPQA	4.6 $\pm$ 0.9	32.1 $\pm$ 2.0	28.6 $\pm$ 1.9	30.3 $\pm$ 2.0	32.1 $\pm$ 2.0	36.5 $\pm$ 2.1
HellaSwag	28.8 $\pm$ 1.4	48.4 $\pm$ 0.5	62.8 $\pm$ 1.5	34.7 $\pm$ 1.5	69.0 $\pm$ 1.5	61.8 $\pm$ 1.5
MMLU	18.6 $\pm$ 1.2	62.1 $\pm$ 0.4	61.1 $\pm$ 1.4	59.7 $\pm$ 1.5	67.8 $\pm$ 1.4	69.8 $\pm$ 1.4
MMLU Pro	7.8 $\pm$ 0.8	31.3 $\pm$ 0.4	37.2 $\pm$ 1.5	34.3 $\pm$ 1.6	40.5 $\pm$ 1.5	43.8 $\pm$ 1.5

Appendix

Table 6: Mean accuracy (

\pm

S.E.) across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) score per benchmark is shown in bold and the second best is underlined .

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	1.7 $\pm$ 0.2	3.0 $\pm$ 0.2	1.2 $\pm$ 0.2	5.5 $\pm$ 0.4	4.3 $\pm$ 0.3	6.7 $\pm$ 0.4
COPA	1.5 $\pm$ 0.7	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	24.8 $\pm$ 2.4	0.0 $\pm$ 0.0	24.8 $\pm$ 2.3
GPQA	1.2 $\pm$ 0.2	10.9 $\pm$ 0.7	11.6 $\pm$ 0.8	15.0 $\pm$ 1.1	12.4 $\pm$ 0.8	11.7 $\pm$ 0.7
HellaSwag	7.3 $\pm$ 0.4	11.9 $\pm$ 0.1	16.4 $\pm$ 0.4	9.6 $\pm$ 0.4	17.8 $\pm$ 0.4	18.2 $\pm$ 0.5
MMLU	6.9 $\pm$ 0.4	42.0 $\pm$ 0.3	49.8 $\pm$ 1.3	55.7 $\pm$ 1.4	52.6 $\pm$ 1.2	47.3 $\pm$ 1.0
MMLU Pro	1.4 $\pm$ 0.2	13.0 $\pm$ 0.2	23.9 $\pm$ 1.1	29.0 $\pm$ 1.4	23.4 $\pm$ 1.0	21.6 $\pm$ 0.9

Appendix

Table 7: Mean confidence-weighted accuracy (

\pm

S.E.) across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) score per benchmark is shown in bold and the second best is underlined .

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	-6.9 $\pm$ 1.5	8.2 $\pm$ 0.6	3.7 $\pm$ 0.7	14.0 $\pm$ 1.3	11.9 $\pm$ 1.2	15.9 $\pm$ 1.8
COPA	3.0 $\pm$ 2.2	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	48.0 $\pm$ 5.9	0.0 $\pm$ 0.0	24.0 $\pm$ 9.0
GPQA	-10.1 $\pm$ 1.8	-35.8 $\pm$ 4.0	-42.8 $\pm$ 3.9	-39.4 $\pm$ 3.9	-35.8 $\pm$ 4.0	-27.0 $\pm$ 4.1
HellaSwag	-40.3 $\pm$ 2.9	27.9 $\pm$ 0.8	49.5 $\pm$ 2.3	28.6 $\pm$ 1.8	40.6 $\pm$ 2.8	23.6 $\pm$ 3.1
MMLU	-40.7 $\pm$ 2.3	24.3 $\pm$ 0.8	22.3 $\pm$ 2.9	19.6 $\pm$ 2.9	35.6 $\pm$ 2.8	39.7 $\pm$ 2.7
MMLU Pro	-58.2 $\pm$ 2.0	-37.3 $\pm$ 0.8	-25.5 $\pm$ 3.0	-31.4 $\pm$ 3.2	-19.0 $\pm$ 3.0	-11.9 $\pm$ 3.1

Appendix

Table 8: Mean ternary score (

\pm

S.E.) across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) score per benchmark is shown in bold and the second best is underlined .

	DialoGPT-Medium	Llama3.2 3B Instruct	Llama TFree HAT Pretrained 7B DPO	Mistral 7B Instruct v0.3	Llama3.1 8B Instruct	DeepSeek R1 0528 Qwen3 8B
ARC	-19.8 $\pm$ 0.3	-12.3 $\pm$ 0.2	-9.9 $\pm$ 0.2	-8.7 $\pm$ 0.3	-11.3 $\pm$ 0.3	-15.9 $\pm$ 0.3
COPA	0.5 $\pm$ 0.3	2.3 $\pm$ 0.3	2.7 $\pm$ 0.3	8.3 $\pm$ 0.8	2.4 $\pm$ 0.3	2.6 $\pm$ 0.9
GPQA	-17.5 $\pm$ 0.4	-36.2 $\pm$ 0.6	-34.9 $\pm$ 0.9	-36.3 $\pm$ 1.3	-34.0 $\pm$ 0.8	-31.9 $\pm$ 0.4
HellaSwag	-27.6 $\pm$ 0.2	-21.3 $\pm$ 0.1	-18.8 $\pm$ 0.2	-15.4 $\pm$ 0.2	-22.6 $\pm$ 0.2	-30.2 $\pm$ 0.2
MMLU	-27.4 $\pm$ 0.4	0.7 $\pm$ 0.4	11.4 $\pm$ 1.7	14.9 $\pm$ 2.1	15.4 $\pm$ 1.5	7.6 $\pm$ 1.3
MMLU Pro	-49.9 $\pm$ 0.3	-46.1 $\pm$ 0.3	-31.5 $\pm$ 1.5	-28.1 $\pm$ 2.1	-31.6 $\pm$ 1.5	-32.7 $\pm$ 1.1

Appendix

Table 9: Mean DCS (

\pm

S.E.) using summed probabilities over 10 abstention phrase variations across tested benchmarks and models. All scores are multiplied by

100

for readability. The best (highest) DCS per benchmark is shown in bold and the second best is underlined .

Explore similar work

Jan 12, 2026cs.CL

Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations

Confidence estimation (CE) indicates how reliable the answers of large language models are and impacts user trust and decision-making. Existing evaluations mainly concern the alignment between confidence and correctness, but ignore the variability of language: confidence estimates should remain consistent under semantically equivalent prompts or answer variations, while changing when answer meaning differs, as this may indicate a change in correctness. Therefore, we introduce a novel evaluation framework based on three complementary properties: \textbf{robustness} to prompt perturbations, \textbf{stability} across semantically equivalent answers, and \textbf{sensitivity} to semantically different answers. We show that these metrics are largely independent from existing CE metrics, and that common CE methods often fail on them: while most methods achieve high robustness and stability, they struggle to distinguish semantically different answers, potentially because they do not effectively leverage generation-side information. Overall, our framework exposes overlooked limitations of current CE evaluations and provides guidance for selecting confidence estimators for real-world applications.

Yuxi Xia, Dennis Ulmer, Terra Blevins +3

Faculty of Computer Science, UniVie Doctoral School Computer Science · Faculty of Philological and Cultural Studies, University of Vienna, Austria · ILLC, University of Amsterdam, Netherlands +2

Confidence Estimation

Ask PDF

Jul 28, 2026cs.AI

Position: Evaluation Scores Are Perishable Knowledge Claims

Evaluation methodologies for language models increasingly combine multiple signals, from automated metrics and LLM-as-judge ratings to human assessments and benchmark suite results. When these signals are aggregated via averaging, evaluation confidence can then substantially exceed the reliability of the weakest signal: a phenomenon we call trust inflation in evaluation. We argue that evaluation scores should be treated as epistemic claims with three properties: formality (human evaluation provides stronger evidence than an automated metric), scope (a benchmark result applies to the tested distribution, not universally), and validity windows (benchmark results expire as contamination accumulates and distributions shift). Several converging research traditions (chain-of-thought analysis, possibilistic logic, and algebraic theory) establish weakest-link aggregation as the conservative endpoint of a parameterized operator family controlled by a single pessimism parameter. Drawing on those traditions, and on concrete lessons from building an evaluation harness for agentic AI, we propose that evaluation results carry explicit metadata (formality tier, scope declaration, and expiration date) to make their epistemic status transparent. We illustrate the cost of mean aggregation on the public HELM leaderboard: across 54 frontier models on ten scenarios, the top-five models ranked by mean score and by weakest-link are completely disjoint.

Sankalp Gilda, Shlok Gilda

DeepThought Solutions · Meta · University of Florida

Confidence Calibration Position

Ask PDF

Sep 17, 2025cs.CL

Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs

Large language models are known to hallucinate, generating linguistically plausible but incorrect answers to questions. Uncertainty quantification has been proposed as a strategy to detect such behaviour, but existing methods lack a unified framework to assess reliability at both the prompt and answer level. We introduce a geometric framework which quantifies language model uncertainty at both levels by explicitly modelling a prompt-conditioned semantic distribution in answer embedding space. Our approach is black-box and sampling-based; we generate multiple answers per prompt, and use archetypal analysis to estimate a geometric support for the answer distribution. At the prompt level, we approximate the distribution entropy to quantify uncertainty; for each individual answer, we then use notions of atypicality to assess its reliability relative to the batch. We employ our framework to not only detect hallucinations but correct them, by selecting the batch example deemed most reliable. Experiments show that our framework performs comparably to or better than prior methods on short form question-answering datasets, and achieves superior results on medical datasets where hallucinations carry particularly critical risks. Beyond pure performance, we suggest the theoretical grounding of our work provides support for semantic distributions as useful objects of study for language model uncertainty.

Edward Phillips, Sean Wu, Soheila Molaei +3

Department of Engineering Science, University of Oxford · GlaxoSmithKline · Oxford Suzhou Centre for Advanced Research

Large Language Model Uncertainty Large Language Model Hallucination Hallucination Mitigation Map Uncertainty Uncertainty Quantification

Ask PDF