LLM Uncertainty Estimation

LLM: Large Language Model

Overview

Large language model uncertainty estimation studies how to tell when an answer, reasoning step, or action may be unreliable, and whether the cause is missing knowledge, ambiguous input, or random variation. Methods include sampling and semantic agreement, token probabilities and entropy, confidence calibration, probes of hidden representations, and Bayesian or evidential models; conformal prediction adds statistical guarantees for abstention or prediction sets. These signals are used to decide when to answer, ask for clarification, retrieve evidence, spend more computation, or defer to a person, with tests spanning factual QA, reasoning, medicine, code, and multimodal tasks. A central challenge is that no signal works reliably across settings: confidence can be miscalibrated or reflect commitment rather than correctness, and agreement may preserve shared errors, so methods need task-specific validation and robust evaluation under ambiguity and distribution shift.

Latest papers

CardsList

416 papers16 papers in the last four weeks, down 67% on the four weeks before. 0.1% of all new papers.

  1. Advancing the Calibration-Compute Frontier of LLM Classifiers via Asymmetric Duos

    Nov 13, 2026Berke Akturk, Ali Can Sahin, Rasit Cem Solatay +3Language Model CalibrationLLM Uncertainty Estimation

  2. Black-Box Uncertainty Quantification for Large Language Models via Ensemble-of-Ensembles

    Nov 13, 2026Wang Ma, Debarun Bhattacharjya, Junkyu Lee +3LLM Uncertainty EstimationUncertainty Decomposition

  3. Where Is Uncertainty Observed? A Mechanistic Understanding of Uncertainty Quantification in LLMs

    Nov 11, 2026Xiaoyu Liang, Virginia Ceccatelli, Aliai Eusebi +4Uncertainty QuantificationLLM Uncertainty Estimation

  4. Free Lunch Uncertainty: Sequence-Level Rényi Entropy from Particle Decoding for Adaptive LLM Reasoning

    Nov 10, 2026Erfan Baghaei Potraghloo, Milad Heidari Sorkhehdizaj, Shahab Rahimirad +6LLM Uncertainty EstimationAdaptive Reasoning in Language Models

  5. Improving Confidence Estimation in Language Models with Linear Representations of Ambiguity

    Nov 9, 2026Themistoklis Nikas, Nishant Subramani, Aaron MuellerConfidence Estimation in Language ModelsLLM Uncertainty Estimation

  6. Distinguishing Extraction Failures and Clinical Discordance with LLM Uncertainty on Radiology Reports

    Nov 7, 2026Jerjes Aguirre-Chavez, Shant Malkasian, Roshun Sankaran +2LLM ReliabilityClinical Information Extraction

  7. Embedding Semantic Entropy: NLI-Free Semantic Uncertainty Estimation with Geometry-Aware Clustering

    Nov 2, 2026Akshiya Raveendran, Praveenasarma Baskarakurukkal, Pragatheeswaran Vipulanandan +2LLM Uncertainty EstimationSemantic Uncertainty Estimation

  8. Specification-Aware Predictive Uncertainty Decomposition

    Nov 2, 2026Roman Vashurin, Mikhail Seleznyov, Assel Yermekova +7Uncertainty DecompositionLLM Uncertainty Estimation

  9. Do LLMs Know When They Are Wrong About Molecules? In-Context Property Prediction with Self-Reported Uncertainty

    Oct 30, 2026Matthias Busch, Marius Tacke, Tamadur Albaraghtheh +5Molecular Property PredictionLLM Evaluation

  10. Neighborhood Elites: Quality-Diversity Search for Uncertainty Quantification in Question Answering

    Oct 29, 2026Mesut Ceylan, Arda Baris Basaran, Elgar Fleisch +1Uncertainty QuantificationLLM Uncertainty Estimation

  11. Distilling the Shape of Uncertainty: What On-Policy Distillation Learns and How to Predict It

    Oct 28, 2026Shiping Yang, Chuanqi Tang, Hossein Hajimirsadeghi +3Language Model DistillationOn-Policy Distillation

  12. What Hallucinations Have We Measured? Semantic Illusion, Confabulation, and Expressed Uncertainty

    Oct 28, 2026Hazel H. Kim, Jakub Binkowski, Ekaterina Fadeeva +23Hallucination in Language ModelsLLM Hallucination Detection

  13. Detect Before You Disambiguate: Perturbation-Based Input Ambiguity Detection in LLMs

    Oct 26, 2026Tiejin Chen, Xiaoou Liu, Michael Clifford +5LLM Uncertainty Estimation

  14. Measuring Mechanistic Uncertainty in Large Language Models via Causal Robustness

    Oct 15, 2026Aneesh Komanduri, Majid Afshar, Timothy A. Miller +1LLM Uncertainty EstimationCausal Interventions in Language Models

  15. LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents

    Oct 15, 2026Qingni Wang, Changdae Oh, Yuzhe Yang +4Computer-Use AgentsAI Agent Monitoring

  16. GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents

    Oct 15, 2026Yaxin Ma, Wenbo Zhang, Satya Krishna Pothapragada +1Behavioral Consistency of LLM AgentsLLM Agent Evaluation

  17. Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation

    Oct 15, 2026Licheng Pan, Haocheng Yang, Haoxuan Li +7Language Model Generation EvaluationRubric-Based Evaluation

  18. Argument Graph Uncertainty: Quantifying Uncertainty from the Logical Structure of Reasoning Chains

    Oct 14, 2026Simon Baur, Sumire Honda, Wojciech Samek +1LLM Uncertainty EstimationUncertainty Calibration

  19. What Is a Clarification? Estimating Model-Dependent Input Ambiguity

    Oct 14, 2026Jingqin Tang, Tim Tomov, Sebastian Schmidt +2LLM Uncertainty Estimation

  20. Gray-Box Hallucination Detection in LLMs with Token Identities

    Oct 14, 2026Shaked Goldshtein, Yitzchak Grunbaum, Fabrizio Frasca +2LLM Hallucination DetectionLLM Uncertainty Estimation

  21. ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertainty

    Oct 12, 2026Yuyang Dai, Yuting Tang, Futoshi Futami +1LLM Uncertainty EstimationProbability Calibration

  22. Autoregressive Decoding as a Martingale: Characterizing the Stochastic Dynamics of LLM Generation

    Oct 12, 2026Xiahui Zhu, Zhenyi Qi, Wanwei Liu +4Autoregressive DecodingAutoregressive Language Modeling

  23. RL-ARC: Calibrating Large Reasoning Models via Reasoning-guided Uncertainty

    Oct 8, 2026Gukhyeon Lee, SangKeun LeeLanguage Model CalibrationLLM Uncertainty Estimation

  24. Characterizing Overconfident Failure in LLM-Based Code Generation

    Oct 8, 2026Ravishka Rathnasuriya, Wei YangLLM Code GenerationConfidence Estimation in Language Models

  25. U-Space: Uncovering When and Why Uncertainty Arises in Language Models

    Oct 6, 2026Tobias Braun, Nils Loose, Alexander Herzog +4LLM Uncertainty EstimationConfidence Estimation in Language Models

  26. Sequential Probabilistic Uncertainty Estimation for Parallel Multi-Agent Reasoning Systems

    Oct 6, 2026Tunyu Zhang, Zihao Zhao, Yusong Zhao +5LLM Uncertainty EstimationMulti-Agent LLM Systems