XU-RS: Explaining Credal Width in Random-Set Language Models

David Achara; Maryam Sultana; Alexander D. Rast; Fabio Cuzzolin

cs.AISep 29, 2026

XU-RS: Explaining Credal Width in Random-Set Language Models

Authors: David Achara, Maryam Sultana, Alexander D. Rast, Fabio Cuzzolin

Organizations: Institute for AI, Data Analysis and Systems (AIDAS) Oxford Brookes University

Abstract

Uncertainty estimates tell us how unsure a model is, but not why. Without knowing which parts of an input influences a model's uncertainty, we cannot tell whether that uncertainty score depends on input features that are relevant for the task. We study this problem in randomset classifiers built using pretrained language models. These classifiers assign probability to individual answers and to groups of answers, producing lower and upper probabilities for each answer; The difference between these probabilities, called credal width, is used to represent epistemic uncertainty about an answer arising from limited training data. We propose XU-RS, a framework that attributes an answer's credal width to the input tokens (words or word pieces) supplied to a language model. XU-RS uses Expected Gradients (a standard feature attribution method) to estimate how input tokens contribute to credal width. The proposed framework is evaluated on a MedQA dataset using SmolLM3-3B and Llama-2-7B models, demonstrating that setting the embedding of a token ranked highly by XU-RS to zero (zero-masking) causes larger changes in credal width than zero-masking randomly selected tokens. In addition, we show that normalisation can cause other answer groups to influence an answer's width, reveal how token attribution can mask numerical errors, and provide diagnostic checks to verify whether a token ranked highly by XU-RS meaningfully explains model uncertainty.

Figures & tables

Model	Questions	Mean advantage	95% CI
SmolLM3-3B	0.0381	[0.0243, 0.0532]
Llama-2-7B	0.0450	[0.0295, 0.0617]

Table 1: Mean advantage in absolute width change from zero-masking the highest-ranked rather than random eligible tokens (Equation 8 ); 100 MedQA questions per model. Intervals are question-level 95% bootstrap intervals.

Reference	Rank $\rho$	$R$	Vector difference
Training prompts	0.9951	0.0652	0.1563
Aligned paired prompt	1.0000	0.0144	0.0384
Zero-embedding baseline	0.9840	0.6428	0.3375

Table 2: Numerical checks on four synthetic cases. Entries are medians. Rank correlation and relative vector difference compare sampling budgets;

R

is the completeness residual (Equation 9 ).

Case	$\Delta_{\mathrm{natural}}$	$\Delta_{\mathrm{aligned}}$	Absolute shift	IG residual
family007_case00	$-0.478592$	$+0.002640$	0.481233	$1.10\times 10^{-5}$
family094_case02	$-0.538493$	$-0.164970$	0.373522	$4.61\times 10^{-7}$
family098_case00	$-0.629888$	$-0.427342$	0.202545	$7.39\times 10^{-6}$
family017_case00	$-0.619253$	$-0.082523$	0.536730	$6.51\times 10^{-6}$

Table 3: Width differences using the original and aligned paired reference prompts (Equation 10 ). Shift is

|\Delta_{\rm natural}-\Delta_{\rm aligned}|

; residuals refer to IG with aligned references. All evaluations use the same FP32 model.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	SmolLM3-3B	Llama-2-7B
LoRA rank	16	8
LoRA scaling parameter	32	16
LoRA dropout	0.05	0.05
LoRA learning rate	$10^{-4}$	$10^{-4}$
Output-layer learning rate	$5\times 10^{-5}$	$10^{-4}$
Weight decay	0.01	0.1

Appendix

Table 4: Training settings for the main LoRA classifiers. Patience is the number of epochs without improvement allowed before stopping. Checkpoints are selected using development-set answer negative log-likelihood.

Setting	SmolLM3-3B	Llama-2-7B
Reference collections	1	3
Prompts per collection	256	1,024
EG samples per collection	512	1,024
Attribution base seed	11	11
Samples processed per call	8	2
Model arithmetic	FP16	BF16

Appendix

Table 5: Settings for the main MedQA attribution runs. FP16 and BF16 are different 16-bit floating-point formats. The samples-per-call setting controls how many samples are processed together.

Measurement	Required value
Median Spearman correlation between section rankings	$\geq 0.90$
Fraction agreeing on the highest-attributed option	$\geq 0.80$
Mean overlap among the 20 highest-ranked tokens	$\geq 0.75$
Median mean absolute difference between section shares	$\leq 0.02$
Maximum difference in the selected answer’s width	$\leq 10^{-6}$

Appendix

Table 6: Criteria used for the final Llama ranking checks. These assess agreement between repeated calculations, not whether the attributions explain meaningful evidence.

Measurement	Required value
Median completeness residual	$\leq 0.01$
Maximum completeness residual	$\leq 0.05$
Median Spearman correlation between token rankings	$\geq 0.95$
Median overlap among the 20 highest-ranked tokens	$\geq 0.80$
Median cosine similarity between attribution vectors	$\geq 0.99$
Median relative Euclidean distance between vectors	$\leq 0.10$

Appendix

Table 7: Criteria for the initial four-example numerical study. Agreement is assessed between attribution calculations using different sample counts and random seeds for the same original prompt and reference collection.

Configuration	Rescaling (%)	Mean total adjustment
SmolLM3: head only	95.84	1.2311
SmolLM3: partial tuning	99.84	2.1623
SmolLM3: LoRA	99.92	1.7221
Llama-2: LoRA, seed 7	99.14	2.2889
Llama-2: LoRA, seed 17	99.61	2.4980
Llama-2: LoRA, seed 23	99.92	2.8108

Appendix

Table 8: Correction on 1,273 test questions per configuration. Rescaling means

s>1+10^{-6}

. Adjustment is the mean total absolute change from intermediate values to final masses. Negative values occur in every tested prediction.

IG reference	Region	Clue share (%)	Difference (pp)	Best rank
Training prompts	All text	2.07	$-1.33$	44
Training prompts	Case text	22.97	$-3.40$	5
Aligned paired prompt	All text	52.59	$+50.89$	1
Aligned paired prompt	Case text	52.59	$+33.02$	1

Appendix

Table 9: Attribution to the distinguishing clue in one example. “Difference” is the clue’s magnitude share minus the mean control-window share, expressed in percentage points. “Best rank” is the rank of the highest-ranked clue token within the selected region; rank one is highest.

Training seed	Training error	Development error
7	0.1221	0.4263
17	0.1176	0.4261
23	0.1173	0.4359

Appendix

Table 10: Mean total absolute error in the predicted answer-set probabilities after the nonlinear model’s 100-epoch schedule. Development values concern wording templates not used during training.

Explore similar work

Reading Calibrated Uncertainty from Language Model Trajectories
May 19, 2026Aliai Eusebi, Alexander Herzog, Xiaoyu Liang +3Large Language Model Uncertainty Calibrated Uncertainty
University College London, London, United Kingdom · Core46, London, United Kingdom
Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs
Sep 29, 2026Feiyang Li, Shengjing Liu, Qi Zhan +7Large Language Model Uncertainty Confidence Calibration
College of Computer Science and Software Engineering, Shenzhen University · Tsinghua University · Behavioral and Spatial AI Lab, Peking University & Tongji University
Credal Large Language Models for Semantic Commitment under Uncertainty
Aug 24, 2026Shireen Kudukkil Manchingal, Sofiia Nikolenko, Fabio CuzzolinLarge Language Model Uncertainty Cryptographic Merkle Tree Commitments
Oxford Dynamics Oxford · Ludwig-Maximilians-Universität München Munich · Institute for Artificial Intelligence, Data Analysis and Systems (AIDAS) School of Engineering Computing & Mathematics Oxford Brookes University, Oxford, UK

May 19, 2026, cs.LGJ/K move · Enter open · S save

Reading Calibrated Uncertainty from Language Model Trajectories

Aliai Eusebi, Alexander Herzog, Xiaoyu Liang, Marie Vasek+2

University College London, London, United Kingdom · Core46, London, United Kingdom

Large Language Model Uncertainty Calibrated Uncertainty Uncertainty Uncertainty Quantification Model Activations Large Language Model Generation+2

Open paper AskPDF arXiv

The maximum softmax probability (MSP) represents a default approach when evaluating uncertainty quantification for language model generation with structured output. Although cheap, it is often miscalibrated. Methods that probe the model's internal activations feed raw hidden states into opaque classifiers, reading activations as static snapshots and leaving implicit the layer-wise trajectory by which a representation is formed. Yet, similar endpoints can arise from very different paths, and how evidence accumulates, reinforces, or reverses across depth might reveal uncertainty that final probabilities obscure. We extract eleven scale-invariant geometric features, tracing the cumulative path of per-layer MLP updates, and feed them to a sparse linear probe. The probe outperforms MSP under selective abstention, with gains scaling with baseline miscalibration up to 21 AURC points. Because every feature has a closed-form geometric meaning, the probe's coefficients trace how and where along depth errors take shape -- which layers commit prematurely, which contradict the running state, where trajectories drift away from their endpoint.

Abstract

Explore similar work

Reading Calibrated Uncertainty from Language Model Trajectories

Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs

Credal Large Language Models for Semantic Commitment under Uncertainty