Semantic Uncertainty Estimation

Latest papers

14 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Semantic Uncertainty Quantification Needs Factual Equivalence
Sep 28, 2026Joseph Hoche, Quentin Guimard, Gianni FranchiConfidence Estimation in Language Models Factual Consistency Evaluation
AMIAD, Pˆole Recherche Palaiseau, France
Using Semantic Uncertainty to Estimate Transition Relevance in Turn-taking
Sep 10, 2026Muhammad Umair, Jan P. de RuiterSpoken Dialogue Systems Turn-Taking Prediction
Department of Computer Science Tufts University Medford, Massachusetts, USA
BiG-SURE - Bipartite Graph for Semantic Uncertainty and Reliability Estimation of LLMs
Aug 31, 2026Debarpan Bhattacharya, Malay Phadke, Sriram GanapathyUncertainty Quantification LLM Uncertainty Estimation
LEAP Lab, Electrical Engineering, Indian Institute of Science · TANUH AI Centre of Excellence, Indian Institute of Science
Attention-Path Fragility as an Uncertainty Signal in Large Language Models
Aug 11, 2026Minsoo Kim, Sungyoung Ji, Kisung Moon +1Confidence Estimation in Language Models LLM Uncertainty Estimation
POSCO Holdings Future Technology Research Institute 440, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systems
Aug 11, 2026John Knowlton, Aritra Guha, Risto MiikkulainenSemantic Uncertainty Estimation Multi-Agent Orchestration
The University of Texas at Austin, USA · AT&T Chief Data Office, USA · Cognizant AI Lab, USA
Calibrating Semantic Uncertainty from Observable Language-Model Probabilities
Jul 20, 2026Matthew F. DixonUncertainty Quantification Confidence Estimation in Language Models
Artificial Intelligence Finance Institute (AIFI), 69 Charlton St, New York, NY 10014, USA
Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?
Jun 30, 2026Ta Duc Huy, Trang Nguyen, Townim Chowdhury +5VLM Evaluation Visual Question Answering
Australian Institute for Machine Learning, Adelaide University, Australia · Flinders University, Australia
LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy
May 5, 2026Hamed Karimi, Vaishali Meyappan, Reza SamaviSelective Prediction Semantic Entropy
Toronto Metropolitan University, Toronto, Ontario, Canada · Vector Institute, Toronto, Ontario, Canada
Mind the Unseen Mass: Unmasking LLM Hallucinations via Soft-Hybrid Alphabet Estimation
Apr 21, 2026Hongxing Pan, Yingying Guo, Wenqing Kuang +1Uncertainty Quantification LLM Hallucination Detection
School of Data Science The Chinese University of Hong Kong, Shenzhen
Improving Semantic Uncertainty Quantification in Language Model Question-Answering via Token-Level Temperature Scaling
Apr 8, 2026Tom A. Lamb, Desi R. Ivanova, Philip H. S. Torr +1Language Model Calibration LLM Uncertainty Estimation
University of Oxford · University of Toronto & Vijil
Sampling-based Task and Kinodynamic Motion Planning under Semantic Uncertainty
Apr 1, 2026Qi Heng Ho, Zachary N. Sunberg, Morteza LahijanianSemantic Uncertainty Estimation Robot Task Planning
Q. H. Ho is with the Kevin T. Crofton Department of Aerospace and Ocean Engineering, Virginia Tech, Blacksburg, VA, USA · Z. N. Sunberg and M. Lahijanian are with the Smead Department of Aerospace Engineering Sciences, University of Colorado Boulder, CO, USA
Semantic Self-Distillation for Language Model Uncertainty
Feb 4, 2026Edward Phillips, Sean Wu, Fredrik K. Gustafsson +2Uncertainty Quantification Confidence Estimation in Language Models
Department of Engineering Science, University of Oxford · Oxford Suzhou Centre for Advanced Research, University of Oxford, Suzhou
Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes
Dec 16, 2025Joseph Hoche, Andrei Bursuc, David Brellmann +4Vision-Language Models Uncertainty Quantification
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · University of Liège +3
Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs
Sep 17, 2025Edward Phillips, Sean Wu, Soheila Molaei +3Uncertainty Quantification LLM Hallucination Mitigation
Department of Engineering Science, University of Oxford · GlaxoSmithKline · Oxford Suzhou Centre for Advanced Research

Sep 28, 2026J/K move · Enter open · S save

Semantic Uncertainty Quantification Needs Factual Equivalence

Joseph Hoche, Quentin Guimard, Gianni Franchi

AMIAD, Pˆole Recherche Palaiseau, France

Confidence Estimation in Language Models Factual Consistency Evaluation LLM Reliability LLM Uncertainty Estimation Semantic Uncertainty Estimation

Open paper AskPDF arXiv

Semantic uncertainty quantification for large language models rests on a common template: sample several answers, measure how much they agree, and treat disagreement as uncertainty. We first formalize this template as two separate roles: an operator that compares two answers, and an aggregator that combines all pairwise comparisons into a scalar. Existing methods differ almost entirely in how they aggregate, while taking the operator off the shelf, typically an NLI model or a generic sentence encoder. We show that this reliance on off-the-shelf operators is the primary bottleneck of semantic UQ: they do not accurately measure factual equivalence of multiple answers to the same question. We resolve this with a deliberately simple recipe: a single encoder trained contrastively to isolate the targeted fact, utilizing synthetic data generated by an LLM and dataset both disjoint from all evaluation settings. Integrating the resulting operator into existing methods improves performance on 120 of 126 evaluation settings (95%) spanning 18 model dataset combinations across language and vision-language models. The best variant reaches 0.76 mean AUROC against 0.68 for the strongest baseline, while replacing the quadratic cross-encoder comparisons of entailment-based operators with one encoder pass per answer. The uniformity of the improvement supports the view that the operator, not the aggregator, is the limiting factor. The same operator also improves single generation token-level estimators: the norm it assigns to each token measures how much that token bears on the answer, and reweighting token log-likelihoods accordingly sharpens the estimate.