VLM Calibration

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 30% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 47

All topics

Mitigating Class-Tail Undercoverage in Medical Vision-Language Models under Clinical Shift
Jul 30, 2026Mushir Akhtar, M. TanveerGroup-Conditional Conformal Prediction Conformal Prediction
Department of Mathematics, Indian Institute of Technology Indore, India
Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation
Jul 24, 2026M M Asif FerdousVLM Robustness Vision-Language Models
Independent Researcher
Confidence Scores in Open-Vocabulary Detection Are a Biased Mixture of Scale and Semantics
Jul 13, 2026Yi Tang Soon, Jun-Wei HsiehOpen-Vocabulary Object Detection Object Detection
Institute of Intelligence Systems, National Yang Ming Chiao Tung University, Tainan, Taiwan · College of Artificial Intelligence and Green Energy, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration
Jul 13, 2026Zheng Zeng, Deepak Sridhar, Nuno VasconcelosVLM Robustness VLM Adaptation
University of California, San Diego, USA
When Prompts Ignore Structure: Graph-Based Attribute Reasoning for Calibrated VLMs
Jul 8, 2026Tanay Sodha, Aditya Sharma, Ramya Hebbalaguppe +2Vision-Language Models VLM Adaptation
Department of Computer Science and Information Systems, Birla Institute of Technology and Science, Pilani, India · TCS Research, New Delhi, India
Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA
Jun 25, 2026Eren Senoglu, Federico Toschi, Nicolo Brunello +2Medical VQA Language Model Calibration
Politecnico di Milano
Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation
Jun 22, 2026Yu Cao, Ziquan Liu, Zhensong Zhang +3VLM Evaluation Physical Consistency in Video Generation
Queen Mary University of London · Huawei Darwin Research Center · Imperial College London
Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Model
Jun 15, 2026Jian Xu, Delu Zeng, John Paisley +1Hallucination Detection in VLMs Active Perception
RIKEN iTHEMS · RIKEN AIP · South China University of Technology +1
FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty
Jun 1, 2026Harry Zhang, Luca CarloneUncertainty Estimation for VLA Models Vision-Language Models
Massachusetts Institute of Technology
Density-Aware Translation of Spurious Correlations in Zero-Shot VLMs
Jun 1, 2026Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie +1VLM Robustness Vision-Language Models
School of Computing and Information Systems, The University of Melbourne, Victoria, Australia.
CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
May 22, 2026Xiaoyi Huang, Kejia Zhang, Zhiming LuoHallucination Detection in VLMs Efficient VLM Inference
Institute of Artificial Intelligence, Xiamen University · Department of Artificial Intelligence, Xiamen University
Ranking-Aware Calibration for Reliable Multimodal Reinforcement Learning
May 16, 2026Peng Cui, Boyao Yang, Jun ZhuMultimodal Robustness VLM Reasoning
Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, Tsinghua-Bosch Joint ML Center, THBI Lab, Tsinghua University, Beijing 100084, China · Dept. of Automation, Tsinghua University, Beijing 100084, China
Grounded or Guessing? LVLM Confidence Estimation via Blind-Image Contrastive Ranking
May 11, 2026Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur +4VLM Evaluation Vision-Language Models
Michigan State University · Independent AI Researcher · JPMorgan AI Research +1
Online Self-Calibration Against Hallucination in Vision-Language Models
May 1, 2026Minghui Chen, Chenxu Yang, Hengjie Zhu +3Vision-Language Models Hallucination in Language Models
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Improving Calibration in Test-Time Prompt Tuning for Vision-Language Models via Data-Free Flatness-Aware Prompt Pretraining
Apr 30, 2026Hyeonseo Jang, Jaebyeong Jeon, Joong-Won Hwang +1Deep Learning Optimization Vision-Language Models
1Yonsei University · 2ETRI
Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
Apr 2, 2026Ji Young Byun, Young-Jin Park, Jean-Philippe Corbeil +1Vision-Language Models VLM Hallucination
Johns Hopkins University, School of Medicine · Massachusetts Institute of Technology · Microsoft Healthcare & Life Sciences
How (Mis)calibrated is your Federated CLIP and what to do about it?
Dec 3, 2025Mainak Singha, Masih Aminbeidokhti, Paolo Casari +3Vision-Language Models VLM Adaptation
University of Trento, Italy · ´Ecole de technologie sup´erieure, QC, Canada · AMIAD, Pˆole Recherche, Palaiseau, France +2

Jul 30, 2026, cs.LGJ/K move · Enter open · S save

Mitigating Class-Tail Undercoverage in Medical Vision-Language Models under Clinical Shift

Mushir Akhtar, M. Tanveer

Department of Mathematics, Indian Institute of Technology Indore, India

Group-Conditional Conformal Prediction Conformal Prediction Medical VLMs VLM Calibration

Open paper AskPDF arXiv

Medical vision-language models (VLMs) can retain high observed marginal coverage after clinical shift while substantially under-covering an individual disease class. The affected class varies with acquisition protocol and backbone geometry, so source prevalence does not reliably reveal the failure. Existing localized and tail-aware conformal methods respectively adapt to test neighborhoods and source-frequency tails, leaving held-out class-wise coverage failure unmodeled. We introduce Class-Tail Adaptive Localized Conformal Deferral (CALCoDe), a post-hoc reliability layer for frozen medical VLMs. Cross-fitted validation predictions identify classes at risk of undercoverage, and a disjoint calibration split estimates their class-conditional tail thresholds. CALCoDe combines each protected threshold with a localized conformal threshold using a one-sided maximum. The resulting set contains every label admitted by the localized rule, with additional protection confined to validation-identified classes. An independently calibrated support audit defers cases with insufficient inlier support. Under exchangeability among accepted examples within each protected class, CALCoDe provides finite-sample coverage at the prespecified guard level and contains the corresponding localized conformal sets; coverage on shifted external cohorts is evaluated empirically. Among standard conformal baselines and recent VLM-specific conformal methods evaluated across two dermatology shifts (HAM10000 to ISIC 2019 and HAM10000 to PAD-UFES-20) and four frozen VLM backbones (BiomedCLIP, OpenAI CLIP ViT-B/32, PubMedCLIP ViT-B/32, and MedSigLIP-448), CALCoDe is the only approach whose observed marginal and worst-class accepted coverage both reach 0.95 in all eight settings. On HAM10000 to ISIC 2019, its average worst-class accepted coverage is 0.970, compared with 0.926 for sTACP and 0.864 for LCP-VLM.

Momentum

Related topics

Latest papers 47