Confidence Estimation in Language Models

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 500% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 134

All topics

Confidence under Visual Token Pruning: Removed Evidence and Risk-Controlled Token Budgets for MLLMs
Apr 13, 2026Kaizhen Tan, Yang Feng, Heqing Du +3VLM Evaluation Confidence Estimation in Language Models
Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA
Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains
Apr 9, 2026Haokai Ma, Lee Yan Zhen, Gang Yang +4RL for Language Models Reinforcement Learning
National University of Singapore, Singapore · Singapore Management University, Singapore
Do LLMs Know What They Know? Measuring Metacognitive Efficiency with Signal Detection Theory
Mar 26, 2026Jon-Paul CacioliConfidence Estimation in Language Models Language Model Calibration
Independent Researcher Melbourne, Australia
On Calibration of Large Language Models: From Response To Capability
Feb 14, 2026Sin-Han Yang, Cheng-Kuang Wu, Chieh-Yen Lin +3LLM Evaluation Confidence Estimation in Language Models
Appier AI Research · National Taiwan University
Semantic Self-Distillation for Language Model Uncertainty
Feb 4, 2026Edward Phillips, Sean Wu, Fredrik K. Gustafsson +2Uncertainty Quantification Confidence Estimation in Language Models
Department of Engineering Science, University of Oxford · Oxford Suzhou Centre for Advanced Research, University of Oxford, Suzhou
Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
Jan 28, 2026Moule Lin, Shuhao Guan, Andrea Patane +2Bayesian Neural Networks Confidence Estimation in Language Models
School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland and Lero the Research Ireland Centre for Software, Ireland · School of Computer Science, University College Dublin, Dublin, Ireland
NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systems
Jan 16, 2026Jiayu Liu, Rui Wang, Qing Zong +9Retrieval-Augmented Generation Confidence Estimation in Language Models
HKUST · UIUC · Northwestern University
Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations
Jan 12, 2026Yuxi Xia, Dennis Ulmer, Terra Blevins +3LLM Evaluation Confidence Estimation in Language Models
Faculty of Computer Science, UniVie Doctoral School Computer Science · Faculty of Philological and Cultural Studies, University of Vienna, Austria · ILLC, University of Amsterdam, Netherlands +2
When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning
Jan 12, 2026Chenjie Hao, Weyl Lu, Yuko Ishiwaka +3Adaptive Inference Confidence Estimation in Language Models
University of California, Davis · SoftBank Corp. · Aizip
Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models
Oct 3, 2025Ej Zhou, Caiqi Zhang, Tiancheng Hu +4Multilingual Language Models Multilingual Language Model Evaluation
Language Technology Lab, University of Cambridge
PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chains
Aug 29, 2025Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema +7Confidence Estimation in Language Models Best-of-N Sampling
Imperial College London · University of Edinburgh · UCL
Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularity
May 23, 2025Shiyu Ni, Keping Bi, Jiafeng Guo +1LLM Hallucination Mitigation Confidence Estimation in Language Models
CAS Key Lab of Network Data Science and Technology, ICT, CAS · State Key Laboratory of AI Safety · University of Chinese Academy of Sciences
Rescaling Confidence: What Scale Design Reveals About LLM Metacognition
Date pendingYuyang Dai, Yuxia WangConfidence Estimation in Language Models Language Model Calibration
INSAIT
ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps
Date pendingJacopo Dardini (University of Bologna), Roberta Calegari (University of Bologna)Uncertainty Quantification Confidence Estimation in Language Models
University of Bologna, Italy

Apr 13, 2026, cs.CVJ/K move · Enter open · S save

Confidence under Visual Token Pruning: Removed Evidence and Risk-Controlled Token Budgets for MLLMs

Kaizhen Tan, Yang Feng, Heqing Du, Hanzhe Hong+2

Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA

VLM Evaluation Confidence Estimation in Language Models Efficient VLM Inference Conformal Risk Control Visual Token Pruning Multimodal Understanding

Open paper AskPDF arXiv

Visual token pruning speeds up multimodal large language models (MLLMs) by keeping a small subset of the visual tokens, and pruning methods are compared by the accuracy they retain. We study what pruning does to the confidence of these models, across common selectors, several MLLMs, and different output formats. Pruning errors concentrate on questions whose evidence the selector removed, and confidence does not register the removal. When the queried object loses all of its tokens, accuracy on these questions drops from 59% to 17%, while confidence stays at the unpruned level. Returning a few object tokens to the kept set recovers most of the lost accuracy. Selectors that keep the most attended tokens remove such evidence most often and produce confident errors, which temperature scaling cannot re-rank. Selectors that avoid keeping redundant tokens stay close to the calibration of the unpruned model. We then use the confidence of the pruned model to set a per-question token budget. The model answers with few tokens first and again with all tokens when its confidence is low. Conformal risk control sets the threshold to bound the expected deviation from the unpruned model. With coverage-based selection, this cascade needs about a third of the prefill tokens of the unpruned model, while with FastV it needs more than the unpruned model. The savings come mainly from how well the confidence ranks the answers that differ from the unpruned ones.

Momentum

Related topics

Latest papers 134