LLM Uncertainty Estimation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 111% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 298

All topics

An Analysis of Training-Free Self-Reported Confidence in Language Models
Sep 17, 2026Lukas Meyer, Sofia Rossi, Wei Chen +2Confidence Estimation in Language Models Language Model Self-Assessment
DreamAI
Intrinsic Sequence-Likelihood Confidence in Retrieval-Dominated Extractive QA: Two Pre-Specified Negatives, and What They Do and Do Not Attribute
Sep 17, 2026Gunwoo Lee, Changmin Sung, Sang-Hwan Gwak +3LLM Evaluation Selective Prediction
Large-scale AI Research Center, Korea Institute of Science and Technology Information (KISTI), 245 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea. · Department of Applied AI, University of Science and Technology (UST), 217 Gajeong-ro, Yuseong-gu, Daejeon, 34113, Republic of Korea.
Chain-of-Thought Entropy as a Reliability Signal: A Preregistered Reproduction
Sep 17, 2026Theodore O. CochranNumerical Reasoning in Language Models LLM Reliability
AI for Altruism
Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
Sep 16, 2026Shardul P. More, Tanuja S. PawarHallucination Detection LLM Hallucination Detection
Rajarambapu Institute of Technology Ishwarpur, Maharashtra, India
Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents
Sep 15, 2026Caiqi Zhang, Xiaochen Zhu, Chengzu Li +3Confidence Estimation in Language Models LLM Uncertainty Estimation
University of Cambridge 1 Solve & Estimate · Google DeepMind
When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control
Sep 15, 2026Ali ŞenolSelective Prediction LLM Prompting
Department of Computer Engineering, Tarsus University, Tarsus, Türkiye
When Confidence Signals Disagree: Local and Global Confidence in Autoregressive Language Models
Sep 15, 2026Julio C. Amador Diaz LopezLLM Evaluation Confidence Estimation in Language Models
SiftyML, LTD London, United Kingdom
R2VC: Modular Fact-Checking with Retrieval, Verification, and Confidence Calibration
Sep 14, 2026Dhruv Dixit, Paritosh PandeyRetrieval-Augmented Generation Automated Fact-Checking
Department of Electrical and Computer Engineering Stevens Institute of Technology Hoboken, New Jersey, United States · Department of Computer Science University of North Carolina Chapel Hill, North Carolina, United States
ProbPlug: A Plugin Uncertainty Network for Reliable Confidence in LLM Binary Classification
Sep 9, 2026Jianzong Wang, Chuhang Liu, Botao Zhao +6Confidence Estimation in Language Models Language Model Calibration
Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China · Tsinghua Shenzhen International Graduate School, Shenzhen, China · Hubei University of Technology, Hubei, China
Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning
Sep 8, 2026Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, Davide Murari +3LLM Uncertainty Estimation LLM Reasoning
University of Cambridge · Harvey Mudd College
From Answers to Interpretations: Rethinking Ambiguity-Induced Aleatoric Uncertainty Estimation in LLMs
Sep 7, 2026Omer Nahum, Niv Nayman, Jonathan Fhima +4LLM Uncertainty Estimation Aleatoric Uncertainty
Data and Decision Sciences, Technion, Israel · Amazon Web Services · Computer Vision Group, University of Bern, Switzerland
From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs
Sep 2, 2026Urja Pawar, Rajitha Ramanayake, Owen O'Neill +5LLM Hallucination Detection LLM Uncertainty Estimation
BNY
DualStake: Dual-Path Confidence Calibration in Deep Research Agents
Sep 1, 2026Yinuo Xu, Yuwei Liang, Jianjie Cheng +4Deep Research Agents Language Model Calibration
NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences (CASIA) · School of Artificial Intelligence, University of Chinese Academy of Sciences · Meituan Inc.
BiG-SURE - Bipartite Graph for Semantic Uncertainty and Reliability Estimation of LLMs
Aug 31, 2026Debarpan Bhattacharya, Malay Phadke, Sriram GanapathyUncertainty Quantification LLM Uncertainty Estimation
LEAP Lab, Electrical Engineering, Indian Institute of Science · TANUH AI Centre of Excellence, Indian Institute of Science
Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents
Aug 30, 2026Zongyue Li, Chengyue Yu, Lei Zang +3Deep Research Agents Long-Horizon Agent Evaluation
Zhejiang University · Ant Group
When Do Supervised UQ Ensembles Improve LLM Hallucination Detection? A Robustness Study
Aug 25, 2026Mohit Singh Chauhan, Vipin Gyanchandani, Dylan BouchardLLM Hallucination Detection LLM Uncertainty Estimation
CVS Health®, Wellesley, MA, USA
Credal Large Language Models for Semantic Commitment under Uncertainty
Aug 24, 2026Shireen Kudukkil Manchingal, Sofiia Nikolenko, Fabio CuzzolinSelective Prediction Confidence Estimation in Language Models
Oxford Dynamics Oxford · Ludwig-Maximilians-Universität München Munich · Institute for Artificial Intelligence, Data Analysis and Systems (AIDAS) School of Engineering Computing & Mathematics Oxford Brookes University, Oxford, UK
PropUQ-MAS: Propagation-Aware Uncertainty Quantification for LLM Multi-Agent Systems
Aug 22, 2026Yaokun Liu, Yifan Liu, Daniel Yue Zhang +3Multi-Agent LLM Systems LLM Uncertainty Estimation
University of Illinois Urbana-Champaign · Scale AI
MAE I Trust Myself? Self-Evaluating VLA Action Generation with Markov Attention Entropy
Aug 17, 2026Aniri, Chen Yilin, Jinhe Bi +7Uncertainty Estimation for VLA Models LLM Uncertainty Estimation
Ludwig Maximilian University of Munich · Munich Center for Machine Learning · East China University of Science and Technology +2
Asymptotic Risk Calibration for Selective Question Answering
Aug 12, 2026Shufan Lin, Sijin DongQuestion Answering LLM Uncertainty Estimation
Zhangjiang University · Ibaraki University
LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence
Aug 12, 2026Po-Jen Ko, Che-Cheng Wu, Hung-Chun Hsu +2Retrieval-Augmented Generation LLM Uncertainty Estimation
1Research Center for Information Technology Innovation, Academia Sinica, Taiwan
Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents
Aug 12, 2026Dylan Bouchard, Mohit Singh ChauhanConfidence Estimation in Language Models LLM Agent Evaluation
When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMs
Aug 11, 2026Utkarsh BahugunaLLM Reliability LLM Uncertainty Estimation
Scaler School of Technology
Attention-Path Fragility as an Uncertainty Signal in Large Language Models
Aug 11, 2026Minsoo Kim, Sungyoung Ji, Kisung Moon +1Confidence Estimation in Language Models LLM Uncertainty Estimation
POSCO Holdings Future Technology Research Institute 440, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
Whisper-Aware LLM: Self-Supervised Uncertainty Learning for Robust Whispered Speech Recognition
Aug 11, 2026Gaopeng Xu, Zhenyu Wang, Zheng Xue +2Uncertainty Quantification Automatic Speech Recognition
Qwen Business Unit of Alibaba, China
Rethinking LLM Verification: Evidence Structure, Uncertainty, and Selective Refinement
Aug 11, 2026Uma Ranjan, Kunal Tilaganji, Aditya Koul +9LLM Grounding LLM Answer Verification
Indian Institute of Technology Jammu · Microsoft Research · SCB Dental College and Hospital, Cuttack
ProbGuard: Calibrated Safety Risk Estimation from LLM Output Distributions
Aug 11, 2026Xinzhe Huang, Biwu Yao, Kedong Xiu +4LLM Guardrails Language Model Calibration
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China · University of Electronic Science and Technology of China, Chengdu, China +3
Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique
Aug 11, 2026Sanidhya Vijayvargiya, Rahul LokeshLLM Hallucination Mitigation Hallucination in Language Models
Samsung Research America
When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information
Aug 10, 2026Maryam Tahermazandarani, Adnan Mahmood, Fahmida Islam +1Hallucination in Language Models LLM Uncertainty Estimation
School of Computing, Macquarie University, Sydney, NSW 2109, Australia
Unsure but Certain: Uncovering the Representation-Confidence Gap in Diffusion Language Models
Aug 9, 2026Saurabh Yadav, Badri Narayana Patro, Vijay Srinivas AgneeswaranConfidence Estimation in Language Models Language Model Calibration
Microsoft India

Sep 17, 2026, cs.CLJ/K move · Enter open · S save

An Analysis of Training-Free Self-Reported Confidence in Language Models

Lukas Meyer, Sofia Rossi, Wei Chen, Thomas Laurent+1

DreamAI

Confidence Estimation in Language Models Language Model Self-Assessment LLM Reliability LLM Uncertainty Estimation Self-Consistency in Language Models

Open paper AskPDF arXiv

Large language models can report a numerical confidence together with generated content, but it is unclear whether this report is more than calibrated rhetoric. We analyze three training-free signals: confidence verbalized with the answer, post-hoc

P(\mathrm{True})

, and agreement with three additional generations on the same 100 TriviaQA questions for two model families. Direct verbalization is a surprisingly strong baseline: after auditing benchmark errors, it reaches AUROC 0.956 and 0.937 for correctness prediction. Three-sample agreement is substantially weaker (0.765 and 0.790), and a fixed interpolation with verbalized confidence has no statistically reliable benefit. Four of nine errors from one model and two of eight from the other receive unanimous sample support, showing that self-consistency can amplify shared misconceptions. Re-eliciting confidence for the same fixed answers with equivalent prompts changes scores by 0.043 to 0.084 on average and flips 4% to 9% of decisions at a 0.8 threshold. An exploratory audit of 100 confidence-tagged biography claims further finds only a modest confidence gap between supported and contradicted claims. These results argue that useful self-reports remain sensitive to elicitation, correlated errors, and benchmark noise.

Momentum

Related topics

Latest papers 298