LLM Uncertainty Estimation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 111% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 298

All topics

Speaking in Self-Assessing Tongues: On the Verbalized Confidence of LLMs in Machine Translation
Jun 15, 2026Ali Marashian, Alexis Palmer, Katharina von der WenseConfidence Estimation in Language Models Language Model Calibration
University of Colorado Boulder, USA · Johannes Gutenberg University Mainz, Germany
Heteroskedastic Signals in Budgeted LLM Verification: Structural Heterogeneity Limits Optimization Gains
Jun 14, 2026Jinlong YangLarge Language Model-Guided Optimization LLM Uncertainty Estimation
Northwestern Polytechnical University
CoRA: Confidence-Rationale Alignment for Reliable Chain-of-Thought Reasoning
Jun 12, 2026Juming Xiong, Weixin Liu, Kevin Guo +9CoT Reasoning RL for Language Model Reasoning
Vanderbilt University · Vanderbilt University Medical Center · Intuit AI Research
Operadic consistency: a label-free signal for compositional reasoning failures in LLMs
Jun 11, 2026Nathaniel Bottman, Yinhong Liu, Kyle RichardsonCompositional Reasoning Reasoning Consistency in Language Models
Incubilate · University of Cambridge · Allen Institute for Artificial Intelligence
LLMs Can Better Capture Human Judgments--With the Right Prompts
Jun 10, 2026Danica Dillion, Chen Cecilia Liu, Baihui Wang +5Human Preference Evaluation LLM Alignment
Complexity Science Hub, Vienna · The Ohio State University · University of Cambridge +2
SAGE: Answer-Conditioned Uncertainty Targets for Verbal Uncertainty Alignment
Jun 9, 2026Kaiwen Shi, Zheyuan Zhang, Yanfang YeConfidence Estimation in Language Models Preference Optimization
University of Notre Dame
Early-Token Confidence Predicts Reasoning Quality in Multi-Agent LLM Debate
Jun 9, 2026Ali Keramati, Justin Cheok, Jacob Horne +1LLM Agent Evaluation Multi-Agent Debate
University of California, Irvine
The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge
Jun 9, 2026Ali Keramati, Justin Cheok, Jacob Horne +1LLM Agent Evaluation Multi-Agent Debate
University of California, Irvine
FASE: Fast Adaptive Semantic Entropy for Code Quality
Jun 8, 2026Shizhe Lin, Ladan TahvildariSemantic Entropy LLM Uncertainty Estimation
University of Waterloo, Canada
Code Is More Than Text: Uncertainty Estimation for Code Generation
Jun 8, 2026Yuling Shi, Caiqi Zhang, Yuexian Li +4Confidence Estimation in Language Models LLM Reliability
Shanghai Jiao Tong University · University of Cambridge
Inference-Time Conformal Reasoning with Valid Factuality Control for Large Language Models
Jun 7, 2026Ting Wang, Yuanjie Shi, Yan Yan +1LLM Inference LLM Reasoning with Graphs
University of Illinois Urbana-Champaign · School of EECS, Washington State University.
Calibration of Structured Ignorance Certificates for Diagnosing Unknown Unknowns in Reasoning Models
Jun 7, 2026Subramanyam SahooHallucination in Language Models Language Model Calibration
Independent Researcher.
The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust
Jun 5, 2026Nishant Subramani, Palash Goyal, Yiwen Song +4Confidence Estimation in Language Models Language Model Calibration
Carnegie Mellon University · Google · Scale AI
Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning
Jun 5, 2026Yijin Zhou, Linqian Zeng, Xiaoya Lu +4Reinforcement Learning LLM Uncertainty Estimation
Shanghai Jiao Tong University, China · Shanghai Artificial Intelligence Laboratory, China · Shanghai Innovation Institute, China
How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures
Jun 4, 2026Tanvi Thoria, Kiana Jafari, Marc R. Schlichting +1Language Model Error Detection LLM Reliability
Department of Computer Science, Stanford University · Department of Aeronautics and Astronautics, Stanford University
CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction
Jun 4, 2026Mohammad Anas Jawad, Cornelia CarageaConfidence Estimation in Language Models Language Model Calibration
Computer Science, University of Illinois Chicago, USA.
Calibrating Overconfidence Without Sacrificing Confidence: Probe-Conditioned Head Intervention for LLMs
Jun 2, 2026Ke Li, Chongzhe Zhang, Zifan Zeng +3Confidence Estimation in Language Models Language Model Calibration
Huawei Heisenberg Research Center · EPFL · TU Berlin +1
Integrating Local and Global Entropy for Uncertainty Quantification in LLMs
Jun 2, 2026Johanne Medina, Tianyi Zhou, Keivin Isufaj +2Uncertainty Quantification Confidence Estimation in Language Models
Qatar Computing Research Institute, HBKU Doha, Qatar · KTH Royal Institute of Technology Stockholm, Sweden
Quantifying Faithful Confidence Expression in Large Reasoning Models
Jun 2, 2026Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu +1LLM Evaluation Confidence Estimation in Language Models
1Yale University
Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models
Jun 2, 2026Qi Cao, Takeshi Kojima, Andrew Gambardella +3LLM Evaluation Uncertainty Quantification
The University of Tokyo, Japan
Large Language Models Are Overconfident in Their Own Responses
Jun 2, 2026Mario Sanz-Guerrero, Manuel Mager, Katharina von der WenseConfidence Estimation in Language Models LLM Prompting
∇Johannes Gutenberg University Mainz, Germany · ♠University of Colorado Boulder, USA
Uncertainty-Aware Clarification in LLM Agents with Information Gain
Jun 2, 2026Mengyi Deng, Zhiwei Li, Xin Li +4Clarification Question Generation Tool-Using Agents
Information Hub, HKUST(GZ) · Jilin University · et al.
The Ghost Annotator: a Framework to Explore Human Label Variation in Content Moderation through Conformal Prediction
Jun 1, 2026Mirko Lai, Alessandra Urbinati, Simona Frenda +2Annotator Disagreement Conformal Prediction
Heriot-Watt University, Edinburgh, Scotland · aequa-tech, Torino, Italy · Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA, USA +2
DECK: A Consistency x Confidence Taxonomy of LLM Hallucinations
Jun 1, 2026Mohit Singh ChauhanLanguage Model Error Detection LLM Hallucination Detection
On the Salience of Low-Probability Tokens for AI-Generated Text Detection: A Multiscale Uncertainty Perspective
Jun 1, 2026Yikai Guo, Bin Wang, Xilai Fan +2AI-Generated Text Detection LLM Uncertainty Estimation
Beijing Institute of Computer Technology and Application, Beijing, China · Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · Southeast University, Nanjing, China +1
The Role of Ambiguity in Error Prediction via Uncertainty Quantification
Jun 1, 2026Ieva Raminta Staliūnaitė, James Bishop, Andreas VlachosUncertainty Quantification LLM Uncertainty Estimation
University of Cambridge · The Alan Turing Institute
Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction
Jun 1, 2026Yujia Tong, Yuxi Wang, Yunyang Wan +3LLM Quantization LLM Evaluation
Wuhan University of Technology · Nanyang Technological University
TriLens: Per-Layer Logit-Lens Entropy for White-Box Hallucination Detection
May 31, 2026Bohan Yang, Yijun Gong, Zhi Zhang +3Language Model Error Detection LLM Reliability
Binjiang Institute of Zhejiang University · Beijing Normal-Hong Kong Baptist University · Great Bay University +1
HypothesisMed: Inference-Time Answer Fusion and Structured Hypothesis-Space Reporting for Biomedical Question Answering
May 31, 2026Md Motaleb Hossen Manik, Ge WangBiomedical QA LLM Uncertainty Estimation
Department of Computer Science Rensselaer Polytechnic Institute Troy, NY, USA · Department of Biomedical Engineering Rensselaer Polytechnic Institute Troy, NY, USA
Sequential statistical inference for Large Language Models: Representation, validity, and monitoring
May 30, 2026Yao XieLanguage Model Calibration LLM Reliability
H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology

Jun 15, 2026, cs.CLJ/K move · Enter open · S save

Speaking in Self-Assessing Tongues: On the Verbalized Confidence of LLMs in Machine Translation

Ali Marashian, Alexis Palmer, Katharina von der Wense

University of Colorado Boulder, USA · Johannes Gutenberg University Mainz, Germany

Confidence Estimation in Language Models Language Model Calibration LLM Uncertainty Estimation MT Evaluation

Open paper AskPDF arXiv

The rapid rise in popularity of large language models (LLMs) for translation calls for a thorough study of the reliability of their confidence in their own outputs. Unlike many generation tasks, translation errors and confidence levels can be useful at different levels of granularity (tokens, words, or spans). Unsupervised approaches based on internal signals like predicted probabilities can be misleading because they reflect certainty among alternatives rather than correctness. In addition, they require access to such internal signals. Here, we devise five verbalized methods of extracting an LLM's per-token confidence without those shortcomings and compare their reliability with that of the model's internal signals of certainty. We evaluate reliability using two forms of alignment: fine-grained error detection and calibration. For both, internal and verbalized methods perform similarly, although results vary by model. Interestingly, we find little to no correlation between internal and verbalized methods.

Momentum

Related topics

Latest papers 298