Confidence Estimation for VLMs

VLM: Vision-Language Model

Latest papers

25 papers1 paper in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models
Sep 16, 2026Jisoo Yang, Jaeho Han, Trung X. Pham +1VLM Evaluation Confidence Estimation in Language Models
Department of Artificial Intelligence, Chung-Ang University, Republic of Korea · Van Lang University, Ho Chi Minh City, Vietnam
E2A-Bench: Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning
Sep 13, 2026Xiaoya Wang, Yutong Xu, Junjie WangConfidence Estimation for VLMs Financial Reasoning Benchmarks
Tsinghua University · Jinan University
Target-Checked Reliability Score Refinement for Video Question Answering
Sep 9, 2026Guoxiang Ren, Rohitash ChandraConfidence Estimation for VLMs Video QA
Transitional Artificial Intelligence Research Group, School of Mathematics and Statistics, UNSW Sydney, NSW 2052, Australia
Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Models
Sep 6, 2026Yangyang Xie, Ke Hao, Jiaqi Liu +2Confidence Estimation for VLMs VLM Calibration
Shanghai Jiao Tong University, Shanghai, China · Medical Image Insights Co. Ltd., Shanghai, China
Respect Your Zero-Shot Uncertainty: Conservative Calibration for Test-Time-Adapted Vision-Language Models
Aug 6, 2026Jingyan Jiang, Yaru Sun, Xiao Chen +5VLM Adaptation Test-Time Adaptation of VLMs
1Shenzhen Technology University · 3Shenzhen University · 2Tsinghua University +1
Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction
Aug 3, 2026Priyashree Roy, Sujitha Martin, Mohammad Rostami +6VLM Evaluation Document Understanding
Amazon Web Services
Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answers
Jul 30, 2026Fexiang Liu, Shiye Wang, Qiang Qiu +1VLM Evaluation Confidence Estimation for VLMs
1State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences
Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation
Jul 27, 2026M M Asif FerdousVLM Robustness Selective Prediction
Independent Researcher
Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation
Jul 24, 2026M M Asif FerdousVLM Robustness Vision-Language Models
Independent Researcher
Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?
Jun 30, 2026Ta Duc Huy, Trang Nguyen, Townim Chowdhury +5VLM Evaluation Visual Question Answering
Australian Institute for Machine Learning, Adelaide University, Australia · Flinders University, Australia
Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models
Jun 16, 2026Logan Mann, Yi Xia, Ajit Saravanan +6VLM Evaluation VLM Robustness
Algoverse AI Research · University of California, Santa Barbara · University of California, Berkeley +1
Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models
Jun 14, 2026Reza Khanmohammadi, Kundan Thind, Mohammad M. GhassemiVLM Evaluation Confidence Estimation for VLMs
Michigan State University East Lansing, MI, USA · Henry Ford Health Detroit, MI, USA
FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty
Jun 1, 2026Harry Zhang, Luca CarloneUncertainty Estimation for VLA Models Vision-Language Models
Massachusetts Institute of Technology
VLAConf: Calibrated Task-Success Confidence for Vision-Language-Action Models
May 28, 2026Dehao Huang, Aoxiang Gu, Chengjie Zhang +5Confidence Estimation for VLMs Uncertainty Estimation for VLA Models
Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China. · Zhongguancun Academy, Beijing, China · National Cybersecurity Academy, Wuhan University, Wuhan, China.
Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
May 26, 2026Joseph Hoche, David Brellmann, Gianni FranchiVision-Language Models Large Vision-Language Models
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · ENSTA Paris
Ranking-Aware Calibration for Reliable Multimodal Reinforcement Learning
May 16, 2026Peng Cui, Boyao Yang, Jun ZhuConfidence Estimation for VLMs Multimodal Robustness
Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, Tsinghua-Bosch Joint ML Center, THBI Lab, Tsinghua University, Beijing 100084, China · Dept. of Automation, Tsinghua University, Beijing 100084, China
Grounded or Guessing? LVLM Confidence Estimation via Blind-Image Contrastive Ranking
May 11, 2026Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur +4VLM Evaluation Confidence Estimation for VLMs
Michigan State University · Independent AI Researcher · JPMorgan AI Research +1
Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits
May 5, 2026Logan Mann, Ajit Saravanan, Ishan Dave +4VLM Evaluation VLM Interpretability
UC Santa Barbara · UC Berkeley · NVIDIA +2
Prompt-Induced Score Variance in Zero-Shot Binary Vision-Language Safety Classification
May 1, 2026Charles Weng, Dingwen Li, Alexander MartinVLM Evaluation Confidence Estimation for VLMs
Johns Hopkins University · Independent
Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelines
Apr 15, 2026Keon Kim, Krish ChelikavadaConfidence Estimation for VLMs Vision-Language Grounding
Om Labs
Confidence under Visual Token Pruning: Removed Evidence and Risk-Controlled Token Budgets for MLLMs
Apr 13, 2026Kaizhen Tan, Yang Feng, Heqing Du +3VLM Evaluation Confidence Estimation for VLMs
Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA
VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning
Apr 10, 2026Wenyi Xiao, Xinchi Xu, Leilei GanVLM Calibration Confidence Estimation for VLMs
Zhejiang University
The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Models
Mar 17, 2026Robert Welch, Emir Konuk, Kevin SmithConfidence Estimation for VLMs VLM Reasoning
KTH Royal Institute of Technology, Stockholm, Sweden · Science for Life Laboratory, Stockholm, Sweden
VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation
Feb 24, 2026Seongheon Park, Changdae Oh, Hyeong Kyu Choi +2Confidence Estimation for VLMs VLM Evaluation
University of Wisconsin-Madison · Nanyang Technological University
Towards Calibrating Prompt Tuning of Vision-Language Models
Feb 22, 2026Ashshak Sharifdeen, Fahad Shamshad, Muhammad Akhtar Munir +6VLM Calibration Prompt Learning for VLMs
Mohamed bin Zayed University of AI · University of Colombo · Michigan State University

Sep 16, 2026J/K move · Enter open · S save

The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models

Jisoo Yang, Jaeho Han, Trung X. Pham, Junyeong Kim

Department of Artificial Intelligence, Chung-Ang University, Republic of Korea · Van Lang University, Ho Chi Minh City, Vietnam

VLM Evaluation Confidence Estimation in Language Models VLM Reasoning VLM Calibration Confidence Estimation for VLMs Verbalized Confidence Estimation

Open paper AskPDF arXiv

A calibrated Vision-Language Model (VLM) can repeatedly self-correct, say "Wait, I should recheck," arrive at the wrong answer, and still report high confidence. We find that this occurs because verbalized confidence is largely trajectory-independent in the VLMs and calibration methods we evaluate. We examine this through three complementary lenses: content variation, token masking, and the model's own hesitation markers. We show that confidence is insufficiently sensitive to what the reasoning trajectory actually contains, and that calibration training can paradoxically worsen this disconnect. Since existing metrics like ECE and AUROC cannot detect this problem, we propose the Trajectory-Grounding Score (TGS) in two complementary forms: TGS-self, which compares confidence with and without access to the model's own trajectory, and TGS-pair, which tests whether the model assigns higher confidence to correct trajectories than to flawed ones along the vision, reasoning, and answer axes. We propose TGS-Bench, a model-agnostic suite spanning 10 benchmarks with controlled good/bad trajectory pairs, and show that conventional calibration rankings diverge from trajectory-grounding rankings, exposing a blind spot in current evaluation practice.