Confidence Estimation for VLMs

VLM: Vision-Language Model

Latest papers

32 papers5 papers in the last four weeks, up 67% on the four weeks before. 0.0% of all new papers.

EO-Jev: Decision Model for Earth Observation
Nov 2, 2026Mohamed Atef Abouelhadid, Diaa Addeen Abuhani, Muhammad Akhtar Munir +2Remote Sensing VLMs Geospatial Foundation Models
Mohamed Bin Zayed University of Artificial Intelligence · Massachusetts Institute of Technology · Linköping University Sweden +1
CalVLA: Shaping Native Confidence in Vision-Language-Action Policies with Outcome Supervision
Oct 30, 2026Boniface Chukwuemeka GodwinFailure Detection for VLA Models Confidence Estimation for VLMs
Carnegie Mellon University Africa, Kigali, Rwanda
Uncovering Shortcut Amplification in Video RLVR through the Lens of Perceptual Uncertainty Collapse
Oct 27, 2026Jiafeng Liang, Xianhao Ou, Tao Ren +6RL for Video Reasoning Reinforcement Learning with Verifiable Rewards
Harbin Institute of Technology · Peking University · Huawei Technologies Co., Ltd +2
PROVE: Probabilistic Visual Grounding via Patch-Level Evidence for Vision-Language Models
Oct 23, 2026Mayank Nautiyal, Tianru Zhang, Li Ju +2Vision-Language Grounding Confidence Estimation for VLMs
Department of Information Technology, Uppsala University, Sweden · Science for Life Laboratory, Uppsala University, Sweden
UniJev: Separating Accuracy from Calibration in Trimodal System One Decision Models
Oct 19, 2026Jianing Wang, Youwen Jian, Derong Wang +5Model Calibration VLM Calibration
Fudan University · Institute of Information Engineering, Chinese Academy of Sciences
VisConf: Quality-Aware Consensus for Best-of-N in Vision-Language Model Reasoning
Oct 17, 2026Muhammad Usman, Muhammad Qasim Ayub, Usman Ahad +1Reasoning for VLMs Confidence Estimation for VLMs
Department of Computer Science, Lahore University of Management Sciences · Department of Electrical Engineering, Lahore University of Management Sciences
The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models
Sep 16, 2026Jisoo Yang, Jaeho Han, Trung X. Pham +1VLM Evaluation Confidence Estimation in Language Models
Department of Artificial Intelligence, Chung-Ang University, Republic of Korea · Van Lang University, Ho Chi Minh City, Vietnam
E2A-Bench: Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning
Sep 13, 2026Xiaoya Wang, Yutong Xu, Junjie WangConfidence Estimation for VLMs Financial Reasoning Benchmarks
Tsinghua University · Jinan University
Target-Checked Reliability Score Refinement for Video Question Answering
Sep 9, 2026Guoxiang Ren, Rohitash ChandraConfidence Estimation for VLMs Video Question Answering
Transitional Artificial Intelligence Research Group, School of Mathematics and Statistics, UNSW Sydney, NSW 2052, Australia
Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Models
Sep 6, 2026Yangyang Xie, Ke Hao, Jiaqi Liu +2Confidence Estimation for VLMs VLM Calibration
Shanghai Jiao Tong University, Shanghai, China · Medical Image Insights Co. Ltd., Shanghai, China
Respect Your Zero-Shot Uncertainty: Conservative Calibration for Test-Time-Adapted Vision-Language Models
Aug 6, 2026Jingyan Jiang, Yaru Sun, Xiao Chen +5VLM Adaptation Test-Time Adaptation of VLMs
1Shenzhen Technology University · 3Shenzhen University · 2Tsinghua University +1
Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction
Aug 3, 2026Priyashree Roy, Sujitha Martin, Mohammad Rostami +6VLM Evaluation Document Understanding
Amazon Web Services
Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answers
Jul 30, 2026Fexiang Liu, Shiye Wang, Qiang Qiu +1VLM Evaluation Confidence Estimation for VLMs
1State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences
Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation
Jul 27, 2026M M Asif FerdousVLM Robustness Selective Prediction
Independent Researcher
Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation
Jul 24, 2026M M Asif FerdousVLM Robustness Vision-Language Models
Independent Researcher
Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?
Jun 30, 2026Ta Duc Huy, Trang Nguyen, Townim Chowdhury +5VLM Evaluation Visual Question Answering
Australian Institute for Machine Learning, Adelaide University, Australia · Flinders University, Australia
Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models
Jun 16, 2026Logan Mann, Yi Xia, Ajit Saravanan +6VLM Evaluation VLM Robustness
Algoverse AI Research · University of California, Santa Barbara · University of California, Berkeley +1
Adapt or Calibrate? How Local Labels Buy Certified Coverage in Medical VQA
Jun 14, 2026Reza Khanmohammadi, Kundan Thind, Mohammad M. GhassemiVLM Evaluation Confidence Estimation for VLMs
Michigan State University East Lansing, MI, USA · Henry Ford Health Detroit, MI, USA
FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty
Jun 1, 2026Harry Zhang, Luca CarloneUncertainty Estimation for VLA Models Vision-Language Models
Massachusetts Institute of Technology
VLAConf: Calibrated Task-Success Confidence for Vision-Language-Action Models
May 28, 2026Dehao Huang, Aoxiang Gu, Chengjie Zhang +5Confidence Estimation for VLMs Uncertainty Estimation for VLA Models
Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China. · Zhongguancun Academy, Beijing, China · National Cybersecurity Academy, Wuhan University, Wuhan, China.
Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
May 26, 2026Joseph Hoche, David Brellmann, Gianni FranchiVision-Language Models Large Vision-Language Models
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · ENSTA Paris
Ranking-Aware Calibration for Reliable Multimodal Reinforcement Learning
May 16, 2026Peng Cui, Boyao Yang, Jun ZhuConfidence Estimation for VLMs Multimodal Robustness
Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, Tsinghua-Bosch Joint ML Center, THBI Lab, Tsinghua University, Beijing 100084, China · Dept. of Automation, Tsinghua University, Beijing 100084, China
Grounded or Guessing? LVLM Confidence Estimation via Blind-Image Contrastive Ranking
May 11, 2026Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur +4VLM Evaluation Confidence Estimation for VLMs
Michigan State University · Independent AI Researcher · JPMorgan AI Research +1
Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits
May 5, 2026Logan Mann, Ajit Saravanan, Ishan Dave +4VLM Evaluation VLM Interpretability
UC Santa Barbara · UC Berkeley · NVIDIA +2
Prompt-Induced Score Variance in Zero-Shot Binary Vision-Language Safety Classification
May 1, 2026Charles Weng, Dingwen Li, Alexander MartinVLM Evaluation Confidence Estimation for VLMs
Johns Hopkins University · Independent
Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelines
Apr 15, 2026Keon Kim, Krish ChelikavadaConfidence Estimation for VLMs Vision-Language Grounding
Om Labs
Confidence under Visual Token Pruning: Removed Evidence and Risk-Controlled Token Budgets for MLLMs
Apr 13, 2026Kaizhen Tan, Yang Feng, Heqing Du +3VLM Evaluation Confidence Estimation for VLMs
Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA
VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning
Apr 10, 2026Wenyi Xiao, Xinchi Xu, Leilei GanVLM Calibration Confidence Estimation for VLMs
Zhejiang University
The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Models
Mar 17, 2026Robert Welch, Emir Konuk, Kevin SmithConfidence Estimation for VLMs VLM Reasoning
KTH Royal Institute of Technology, Stockholm, Sweden · Science for Life Laboratory, Stockholm, Sweden
VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation
Feb 24, 2026Seongheon Park, Changdae Oh, Hyeong Kyu Choi +2Confidence Estimation for VLMs VLM Evaluation
University of Wisconsin-Madison · Nanyang Technological University

Nov 2, 2026J/K move · Enter open · S save

EO-Jev: Decision Model for Earth Observation

Mohamed Atef Abouelhadid, Diaa Addeen Abuhani, Muhammad Akhtar Munir, Fahad Shahbaz Khan+1

Mohamed Bin Zayed University of Artificial Intelligence · Massachusetts Institute of Technology · Linköping University Sweden+1

Remote Sensing VLMs Geospatial Foundation Models Confidence Estimation for VLMs

Open paper AskPDF arXiv

Earth observation (EO) workflows require selecting among known outcomes, estimating confidence, and processing large volumes of imagery. Generative vision-language models can address these tasks, but autoregressive decoding is computationally costly and their confidence estimates are unreliable. Decision models offer an alternative, returning probabilities over candidate answers in a single forward pass. However, their accuracy, calibration, and transfer across EO tasks remain insufficiently understood. We investigate these properties through a diagnostic study of d3-flash (8.4B parameters) across 30 settings from 23 public EO benchmarks. We find that compositional reasoning transfers well and calibration improves over its Qwen3.5-9B backbone on 24 settings, while counting, change quantification, damage grading, and absent-object recognition remain challenging. To address these limitations, we adapt d3-flash through low-rank fine-tuning on public EO training splits, producing EO-Jev. EO-Jev improves significantly over d3-flash on 26 of 30 settings and matches or exceeds specialist references on 20 of 28. It also achieves an average gain of 8.0 points across 12 held-out settings and lower expected calibration error than specialists on 21 of 28 settings. These improvements come at a modest cost of 1.8 points in general decision performance. Our results demonstrate the potential of adapting general decision models into competitive EO generalists while highlighting persistent challenges in quantitative perception, option-order consistency, and calibration under distribution shift.