Selective Prediction for VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

11 papers0 papers in the last four weeks, down 100% on the four weeks before. 0.0% of all new papers.

Evidence-Order Calibration for Selective Visual Reasoning under Progressive Loss of Question-Critical Evidence
Aug 29, 2026Muhamathu Ameer Ali Aacaas MuhamathSelective Prediction for VLMs VLM Calibration
Department of Electrical Engineering University of Moratuwa Moratuwa, Sri Lanka
Answer-Level Trust Selection for Physical Vision-Language Reasoning
Aug 20, 2026Rongyu Yu, Ke Niu, Fengxiang HeSelective Prediction for VLMs VLM Evaluation
University of Edinburgh · Fudan University
EVADE: Evidence-Verified Agentic Diagnosis with Escape
Aug 19, 2026Mohaimenul Azam Khan Raiaan, Nur Mohammad FahadSelective Prediction for VLMs Medical VLMs
School of Engineering and Energy, Murdoch University, Murdoch, WA 6150, Australia
MonitorVLM-v2: A Deployed Vision-Language Framework for Real-Time Safety Violation Detection
Aug 2, 2026Jiang Wu, Sichao Wu, Yinsong Ma +2Vision-Language Models Efficient VLM Inference
School of Mechanical Engineering, University of Science and Technology Beijing · The Laboratory for Computational Sensing and Robotics, Johns Hopkins University
When Thinking Hurts: Epistemic Signals in the Reasoning Chains of Visual Language Models
Jul 9, 2026Mayank SingalVLM Evaluation Uncertainty Estimation for VLA Models
Independent Researcher.
Detect Before You Leap: Mirage Detection in Vision-Language Models
May 29, 2026Md. Shaown Miah, S. M. Taiabul Haque, Syed Ishtiaque Ahmed +1Hallucination Detection in VLMs Vision-Language Models
Indiana University Indianapolis · University of Toronto
Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?
May 28, 2026Yue Zhang, Zun Wang, Han Lin +3VLM Evaluation Spatial Reasoning Benchmarks
UNC Chapel Hill · Google Research
SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring
Apr 28, 2026Hector G. Rodriguez, Marcus RohrbachVisual Question Answering VLM Robustness
TU Darmstadt & hessian.AI, Germany
Don't Blink: Evidence Collapse during Multimodal Reasoning
Apr 5, 2026Suresh Raghu, Satwik PandeySelective Prediction for VLMs Vision-Language Models
Independent Researcher, New Delhi, India
Geometric Risk Control for Vision-Language Model OCR
Mar 20, 2026Weile Gong, Zijian Lu, Mingcai Chen +3Selective Prediction for VLMs VLM Hallucination Mitigation
Nanjing University of Posts and Telecommunications, Nanjing, China
To See or To Please: Uncovering Visual Sycophancy and Split Beliefs in VLMs
Mar 19, 2026Rui Hong, Shuxue QuanSelective Prediction for VLMs Sycophancy in VLMs
George Mason University, Fairfax, VA, USA · Independent Researcher

Aug 29, 2026J/K move · Enter open · S save

Evidence-Order Calibration for Selective Visual Reasoning under Progressive Loss of Question-Critical Evidence

Muhamathu Ameer Ali Aacaas Muhamath

Department of Electrical Engineering University of Moratuwa Moratuwa, Sri Lanka

Selective Prediction for VLMs VLM Calibration Vision-Language Models Post-Hoc Calibration Question Answering

Open paper AskPDF arXiv

Vision-language model (VLM) confidence may change in aggregate when visual evidence is degraded while remaining structurally inconsistent within individual examples. We study answer-level reliability along five-step, question-conditioned evidence-loss trajectories. Using a frozen Qwen2.5-VL-3B-Instruct model, we construct 176 accepted GQA-derived trajectories (880 masking conditions) by progressively masking scene-graph-localized question-critical regions. Native sequence confidence has an evidence monotonicity violation rate (EMVR) of 0.436, and 92.0% of trajectories contain at least one adjacent violation. A matched non-critical-region control shows that full critical masking reduces accuracy by 28.2 percentage points, compared with 0.6 points for equally sized non-critical masks; the paired difference is 27.6 points (95% CI [20.0, 34.7]). We train a lightweight post-hoc reliability head on frozen hidden states, sequence confidence, and entropy. Adding evidence-order supervision to binary cross-entropy (BCE) reduces masking EMVR from 0.330 to 0.303 (paired difference -0.027, 95% CI [-0.044, -0.010]). The same mask-trained objective reduces EMVR from 0.449 to 0.402 on held-out question IDs under unseen local Gaussian blur (difference -0.0468, 95% CI [-0.0739, -0.0199]). AUROC, Brier, and AURC differences between the two learned heads are statistically inconclusive, and native confidence remains stronger for selective-risk ranking. The results separate evidence-order consistency from conventional correctness discrimination rather than establishing generic confidence superiority.