cs.CLNov 11, 2026

Where Is Uncertainty Observed? A Mechanistic Understanding of Uncertainty Quantification in LLMs

Authors: Xiaoyu Liang, Virginia Ceccatelli, Aliai Eusebi, Tobias Braun, Nils Loose, Carlo Malagnino, Lorenzo Cavallaro

Organizations: University College London · Technische Universität Darmstadt · Universität zu Lübeck · Mohamed bin Zayed University of Artificial Intelligence

Abstract

Knowing when a large language model's (LLM's) response can be trusted is critical for its safe use. Uncertainty quantification (UQ) can help LLMs recognize if their responses may be unreliable. Existing taxonomies organize UQ methods by uncertainty source or estimation mechanism. They leave unresolved the question of where uncertainty is observed in the model's computation. The answer matters because the observation point determines whether an estimator has access to enough information to recover a given target, and how well it can recover it. Drawing on mechanistic interpretability, we define four mechanistic observation lenses: the structural lens observes component updates, the representational lens observes hidden states, the distributional lens observes token probabilities, and the semantical lens observes distributions over meanings. We relate these lenses through the residual stream and order them by the information they carry, which gives a principle for selecting UQ methods for different targets. We survey existing methods through the four lenses. Furthermore, we distinguish evaluation targets, criteria, and metrics to clarify different UQ evaluation protocols. Finally, we identify open problems and research directions to connect uncertainty estimation to the internal computation and downstream use of LLMs.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering

    May 19, 2026Tiejin Chen, Longchao Da, Xiaoou Liu +1LLM EvaluationUncertainty Quantification

  2. Why Don't You Know? Evaluating the Impact of Uncertainty Sources on Uncertainty Quantification in LLMs

    Apr 12, 2026Maiya Goloburda, Roman Vashurin, Fedor Chernogorskii +4Uncertainty QuantificationLLM Uncertainty Estimation

  3. Improved Confidence Estimates for Black-Box Large Language Models

    Aug 19, 2026Sokhna Diarra Mbacke, Mouloud Belbahri, Gabriel Loaiza-GanemConfidence Estimation in Language ModelsConfidence Estimation