LLM Uncertainty Estimation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 111% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 298

All topics

What Am I Missing? Question-Answering as Hidden State Probing
May 29, 2026Chu Fei Luo, Samuel Dahan, Xiaodan ZhuLLM Self-Refinement Language Model Self-Assessment
Department of Electrical and Computer Engineering & Ingenuity Labs, Queen’s University · Conflict Analytics Lab, Queen’s University · Cornell Law School +1
Shared Doubt: Zero-Shot Cross-Lingual Confidence Estimation for Language Models
May 29, 2026Athina Kyriakou, Dennis Ulmer, Ivan TitovMultilingual Language Models Zero-Shot Learning
° ILCC, University of Edinburgh · M ILLC, University of Amsterdam
Fine-Tuning Improves Information Conveyance in Language Models
May 29, 2026Yuwei Cheng, Weiyi Tian, Haifeng XuLanguage Model Generation Evaluation LLM Fine-Tuning
Department of Data Science University of Chicago Chicago, IL 60637 · Department of Computer Science University of Chicago Chicago, IL 60637
Human-Alignment, Calibration, and Activation Patterns in Large Language Model Uncertainty
May 29, 2026Kyle Moore, Jesse Roberts, Daryl Watson +2Confidence Estimation in Language Models LLM Interpretability
1Vanderbilt University · 2Tennessee Technological University
Improving Selective Classification with Pairwise Queries for Binary Classification
May 28, 2026Harsh Vardhan, Sunav Choudhary, Natwar Modani +1Pairwise Comparison Selective Prediction
CSE, UCSD, 9500 Gilman Drive, La Jolla, 92092, CA, USA. · Adobe Research, 345 Park Ave, San Jose, 95110, CA, USA. · Adobe Research, Marathahalli, Bengaluru, 560087, Karnataka, India. +1
On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
May 28, 2026Daniel Dold, Emanuel Sommer, Julius Kobialka +2Mode Connectivity Bayesian Inference
HTWG Konstanz · LMU Munich · Munich Center for Machine Learning (MCML)
Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective
May 28, 2026Shenghao Ye, Yuxiang Wang, Yu Guo +3Cost-Aware Inference LLM Uncertainty Estimation
University of Science and Technology of China · The University of Melbourne · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence?
May 27, 2026Gabrielle Kaili-May Liu, Arman CohanConfidence Estimation in Language Models Language Model Calibration
Yale University
Reverse Probing: Supervised Token-level Uncertainty Quantification for Large Language Models in Clinical Text
May 27, 2026Bushi Xiao, Sarvesh Soni, Daisy Zhe WangText Summarization LLM Uncertainty Estimation
University of Florida Gainesville, FL · U.S. National Library of Medicine Bethesda, MD
Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantification
May 27, 2026Dylan Bouchard, Mohit Singh Chauhan, Zeya Ahmad +1Language Model Calibration LLM Uncertainty Estimation
CVS Health®, Wellesley, MA, USA
Localizing Input Uncertainty Quantification for Large Language Models via Shapley Values
May 27, 2026Seongjun Lee, Suwan Yoon, Changhee LeeUncertainty Quantification Shapley Value Attribution
Department of Artificial Intelligence, Korea University
Do Models Know Why They Changed Their Mind? Interpretability and Faithfulness of Chain-of-Thought Under Knowledge Conflict
May 26, 2026Pruthvinath Jeripity VenkataCoT Faithfulness Faithfulness of Language Model Explanations
Independent Researcher
Asking Is Not Enough: Protocol Sensitivity in LLM Confidence Calibration
May 26, 2026Hankyeol Kim, Pilsung KangLLM Evaluation Confidence Estimation in Language Models
Seoul National University
Soft Specialists: $α$ -Rényi Ensembles for Uncertainty-Aware LLM Post-Training
May 26, 2026Paula Cordero-Encinar, Georgy Tyukin, Andrew B. DuncanLLM Uncertainty Estimation Language Model Ensembles
Department of Mathematics, Imperial College London · Bessemer AI
Prefix-Safe Bayesian Belief Tracking for LLM Reasoning Reliability:Separating Calibration from Ranking
May 26, 2026Zhenghan Song, Yunyi Li, Yulong LiuBayesian Filtering Language Model Calibration
Cornell University · Columbia University
Aligning LLMs with Human Uncertainty: A Beta-Bernoulli Calibrator for LLM Forecasting
May 26, 2026Hui Dai, Ryan Teehan, Parsa Torabian +1Post-Hoc Calibration Probabilistic Forecasting
Agentic Learning AI Lab, New York University · Chronologies AI
Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models
May 26, 2026Liyi Zhang, Akshay K. Jagadish, Brenden M. Lake +1Language Model Calibration LLM Uncertainty Estimation
Department of Computer Science Princeton University · Princeton AI Lab Princeton University · Department of Psychology and Computer Science Princeton University
Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
May 26, 2026Joseph Hoche, David Brellmann, Gianni FranchiVision-Language Models Large Vision-Language Models
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · ENSTA Paris
Evaluating the Relevance of Uncertainty Estimators for LLM Hallucination
May 26, 2026Yedidia Agnimo, Anna Korba, Annabelle Blangero +2LLM Evaluation Hallucination in Language Models
Ekimetrics · Centre Inria de l’Université Grenoble Alpes · CREST, ENSAE +1
Helicase: Uncertainty-Guided Supply Chain Knowledge Graph Construction with Autonomous Multi-Agent LLMs
May 26, 2026Yunbo Long, Haolang Zhao, Ge Zheng +1Multi-Hop QA KG Construction
Department of Engineering, University of Cambridge, Cambridge, UK · The Alan Turing Institute, London, UK
Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals
May 25, 2026Federico Torrielli, Peter Schneider-Kamp, Lukas Galke PoechConfidence Estimation in Language Models LLM Interpretability
University of Turin · University of Southern Denmark
Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation
May 25, 2026Joris Baan, Wilker Aziz, Barbara Plank +1Clarification Question Generation Question Answering
University of Amsterdam · LMU Munich · European Commission, Joint Research Centre (JRC), Brussels
From Simulation to Enaction: Post-trained language models recognize and react to their own generations
May 25, 2026Asvin G., Jack LindseyLanguage Model Introspection LLM Uncertainty Estimation
Institute for Advanced Study, Princeton · Anthropic
Second Guess: Detecting Uncertainty Through Abstention and Answer Stability in Small Language Models
May 25, 2026Ashwath Vaithinathan Aravindan, Mayank KejriwalConfidence Estimation in Language Models Multiple-Choice Question Answering
University of Southern California Los Angeles, 90007, California, United States of America · Information Sciences Institute 4676 Admiralty Way #1001, Los Angeles, 90292, California, United States of America
Decision-Making with Lightweight Confidence-Aware Language Model for Autonomous Driving
May 25, 2026Ruoyu Yao, Ruiguo Zhong, Pei Liu +3Confidence Estimation in Language Models Language Model Distillation
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China
Inference Time Optimization with Confidence Dynamics
May 24, 2026Yu Wang, Minghao Liu, Jiayun Wang +3Inference-Time Optimization Confidence Estimation in Language Models
Center for Advanced AI, Accenture.
Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction
May 24, 2026João Sedoc, Baotong Zhang, Dean FosterLLM Answer Verification Selective Prediction
New York University
Understanding and Mitigating Premature Confidence for Better LLM Reasoning
May 23, 2026Jingchu Gai, Guanning Zeng, Christina Baek +4Confidence Estimation in Language Models RL for Language Model Reasoning
1Carnegie Mellon University · 2Tsinghua University
LLMs Show No Signs Of Individuated Metacognition
May 22, 2026M. Moran, Mark WhitingLLM Evaluation Confidence Estimation in Language Models
Pareto AI
Breaking the Chains of Probability: Neutrosophic Logic as a New Framework for Epistemic Uncertainty in Large Language Models
May 22, 2026Maikel Yelandi Leyva-Vázquez, Florentin SmarandacheLLM Evaluation Knowledge Conflicts in Language Models
Universidad Bolivariana del Ecuador, Coordinación Académica de Posgrado, Durán, Ecuador · Universidad de Guayaquil, Guayaquil, Ecuador · Universidad Bernardo O’Higgins, Santiago, Chile +1

May 29, 2026, cs.CLJ/K move · Enter open · S save

What Am I Missing? Question-Answering as Hidden State Probing

Chu Fei Luo, Samuel Dahan, Xiaodan Zhu

Department of Electrical and Computer Engineering & Ingenuity Labs, Queen’s University · Conflict Analytics Lab, Queen’s University · Cornell Law School+1

LLM Self-Refinement Language Model Self-Assessment LLM Uncertainty Estimation Metacognition in Language Models Language Model Probing LLM Reasoning

Open paper AskPDF arXiv

Test-time reasoning has become a significant field of study since the introduction of chain-of-thought reasoning in large language models (LLMs). However, the mechanisms of this reasoning process are still under-explored -- from the same input prompt, and even the same partial solution, LLMs can produce varied answers if sampled multiple times. We propose to leverage question-asking as an inference-time intervention that articulates information about the model's hidden state. To achieve that, we present a student-teacher setting where a student asks questions to a teacher. We train a probe on the student's hidden state before and after asking a question and find it is predictive of the trajectory's final correctness, even before generating the teacher's answer. This suggests there is a meaningful signal from the self-diagnosis that occurs during question generation rather than information transfer from the teacher. We then frame question-asking as a sequential decision problem, using this probe as a quality score, and define a gating policy to ask questions that maximize likelihood of correctness. We find that the success of question-asking as an intervention is largely dependent on the model's self-consistency. Our empirical results show a gap between detection and recovery; while our gating policy captures model correctness and uncertainty, interventions are equally likely to harm correct trajectories as they are to recover incorrect ones. This gap between diagnosis and correction has broader implications on language models' capacity for self-refinement under uncertainty.

Momentum

Related topics

Latest papers 298