Large language models (LLMs) are increasingly used in critical domains, where their susceptibility to hallucinations makes reliable and interpretable uncertainty quantification essential. We introduce the concept of mechanistic uncertainty, which characterizes uncertainty through the stability of the internal mechanisms supporting a model's response. We propose the causal robustness margin, defined as the minimum internal intervention required to deviate the model's output, and operationalize it using Jacobian Lens (Jlens) concept steering vectors. Across three open-source models and five question-answering datasets, we show incorrect and uncertain responses exhibit smaller margins, indicating greater sensitivity to internal interventions, and the causal robustness margin complements conventional output-based uncertainty methods. Finally, in a controlled knowledge injection experiment, correctly learned facts show increased margins after training, suggesting that causal robustness provides a mechanistically grounded signal of uncertainty.
Figures & tables
Figure 1: We perform different magnitudes of steering α (both positive and negative) along Jlens consensus token vectors to determine mechanistic uncertainty. Response flips at lower magnitude steers, as shown in the example, indicate high mechanistic uncertainty. We use mid to last layer activations due to informative verbalizable tokens, whereas earlier layers contain a lot of noise.
Figure 2: α∗ plotted against accuracy. Binning is performed at each α∗ value, and average accuracy is computed over each bin. The trend shows accuracy increasing with α∗ .
Dataset
Model
AUROC
αˉcorr∗
αˉinc∗
TruthfulQA
Llama-3.1-8B
57.90
0.182
0.137
Qwen2.5-7B
57.70
0.163
0.128
Gemma-3-12B
57.30
0.148
0.112
TriviaQA
Llama-3.1-8B
74.28
0.1512
0.0474
Qwen2.5-7B
75.14
0.0860
0.0328
Gemma-3-12B
71.08
0.0963
0.0262
Table 1: Causal robustness margins and error discrimination. αˉcorr∗ and αˉinc∗ are mean margins for correct and incorrect responses; larger margins indicate greater robustness. AUROC uses −α∗ to detect incorrect responses.
Figure 3: Correlation between the causal robustness margin α∗ and conventional uncertainty measures across TriviaQA, PopQA, SciQ, MedQA, and TruthfulQA. Each panel shows binned uncertainty statistics as a function of α∗ for Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-3-12B-Instruct. Shaded regions indicate variability within each α∗ bin, and the reported ρ values denote Spearman rank correlation.
Llama-3.1-8B-Instruct
Qwen2.5-7B-Instruct
Gemma-3-12B-Instruct
Dataset
Baseline
Base
+α∗
Base
+α∗
Base
+α∗
TruthfulQA
Self-consistency
0.524
0.569
0.535
0.597
0.440
0.550
Max token prob.
0.505
0.564
0.501
0.574
0.512
0.566
Token entropy
0.538
0.576
0.500
0.567
0.508
0.566
TriviaQA
Self-consistency
0.833
0.847
0.769
0.808
0.653
0.752
Max token prob.
0.828
0.840
0.810
0.822
0.761
0.793
Table 2: AUROC for detecting incorrect answers (positive class: incorrect) for each baseline alone (Base) and combined with α∗ ( +α∗ ) in a logistic regression, using out-of-fold predictions from 5-fold cross-validation averaged over 10 shuffles. Bold : improvement significant under a paired bootstrap over questions (95% CI of the difference excludes 0; 1,000 resamples).
Table 3: Effect of knowledge injection SFT on PopQA accuracy and causal robustness. (a) Accuracy and mean causal robustness margins before and after SFT. (b) Mean margins by correctness transition ( W : wrong; R : right). ΔM=αˉpost∗−αˉpre∗ ; positive values indicate increased robustness. Only target entities are used for SFT.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Prompt for contextualizing with Qwen3-32B
Figure 5: Prompt for generating paraphrases with Qwen3-32B
Dataset
Model
Baseline Acc.
Recovered (%)
Oracle Acc.
TruthfulQA
Llama-3.1-8B
50.6%
24.8%
62.8%
Qwen2.5-7B
64.5%
19.7%
71.5%
Gemma-3-12B
57.5%
17.3%
64.9%
TriviaQA
Llama-3.1-8B
72.8%
24.6%
79.5%
Qwen2.5-7B
55.9%
16.8%
63.3%
Gemma-3-12B
72.3%
12.3%
75.7%
Appendix
Table 4: Baseline accuracy and recovery after steering at the causal robustness margin α∗ . Recovered is the percentage of initially incorrect responses made correct. Oracle-steered accuracy preserves initially correct responses and steers only incorrect responses, requiring ground-truth correctness.
Figure 6: Correlation between the causal robustness margin α∗ and sequence likelihood, perplexity, energy, and semantic entropy across models and datasets.
Understanding why a large language model (LLM) is uncertain about the response is important for their reliable deployment. Current approaches, which either provide a single uncertainty score or rely on the classical aleatoric-epistemic dichotomy, fail to offer actionable insights for improving the generative model. Recent studies have also shown that such methods are not enough for understanding uncertainty in LLMs. In this work, we advocate for an uncertainty decomposition framework that dissects LLM uncertainty into three distinct semantic components: (i) input ambiguity, arising from ambiguous prompts; (ii) knowledge gaps, caused by insufficient parametric evidence; and (iii) decoding randomness, stemming from stochastic sampling. Through a series of experiments we demonstrate that the dominance of these components can shift across model size and task. Our framework provides a better understanding to audit LLM reliability and detect hallucinations, paving the way for targeted interventions and more trustworthy systems.
Large Language Models (LLMs) that can express interpretable and calibrated uncertainty are crucial in high-stakes domains. While methods to compute uncertainty post-hoc exist, they are often sampling-based and therefore computationally expensive or lack calibration. We propose a three-stage pipeline to post-train LLMs to efficiently infer calibrated uncertainty estimates for their responses. First, we compute fine-grained entropy-based uncertainty scores on the training data, capturing the distributional variability of model outputs in embedding space. Second, these scores are calibrated via Platt scaling, producing reliable and human-interpretable uncertainty signals. Finally, the target LLM is post-trained via reinforcement learning to align its policy with these calibrated signals through a verifiable reward function. Unlike post-hoc uncertainty estimation methods, our approach provides interpretable and computationally efficient uncertainty estimates at test time. Experiments show that models trained with our pipeline achieve better calibration than baselines and generalize to unseen tasks without further processing, suggesting that they learn a robust uncertainty reasoning behavior.
Azza Jenane, Nassim Walha, Lukas Kuhn +1
German Cancer Research Center (DKFZ) German Cancer Consortium (DKTK) Goethe University Frankfurt, Germany
Large Language Models (LLMs) have achieved significant breakthroughs across various domains, but they can still produce unreliable or misleading outputs. For responsible LLM applications, uncertainty quantification techniques are used to estimate a model's uncertainty about its outputs, indicating the likelihood that those outputs may be problematic. For LLM reasoning tasks, it is essential to estimate uncertainty not only in the final answer but also in the intermediate reasoning process, particularly to identify where uncertainty arises. Such information may enable more fine-grained and targeted interventions during inference. In this study, we investigate which metrics can effectively localize uncertain places within an LLM reasoning trajectory. Our study reveals that uncertain intermediate continuations are more likely to occur at tokens that are highly sensitive to perturbations in the embeddings of preceding tokens. In our experiments, we show that such perturbation-based metrics achieve stronger performance in localizing uncertain intermediate steps than baseline methods, including probability-based, sampling-based, and Bayesian-based approaches. Meanwhile, our proposed metrics also enjoy good simplicity and efficiency.