VLM Calibration

VLM: Vision-Language Model

Overview

Vision-language model calibration studies whether confidence in image-text predictions reflects their actual correctness, supporting safer decisions such as abstaining or requesting more evidence. Approaches include temperature and logit scaling, prompt and embedding calibration, conformal prediction, confidence probes, and training or decoding methods that use visual evidence to detect hallucinations. These methods are evaluated across zero-shot classification, visual question answering, detection, and specialized settings such as medical imaging, documents, and autonomous driving, including under distribution shift. Calibration can improve selective prediction, but standard metrics may not reveal whether confidence tracks visual grounding or sound reasoning, and reliability can break under severe shifts or deliberate perturbations.

Latest papers

CardsList

91 papers11 papers in the last four weeks, up 83% on the four weeks before. 0.1% of all new papers.

  1. Calibrate Once, Grow the Vocabulary: Candidate-Independent Scores for Conformal Risk Control in Open-Vocabulary Segmentation

    Nov 16, 2026Md Tanveer Hossain Munim, Bijoy Ahmed Saiem, Muhammad Abdullah AdnanConformal Risk ControlOpen-Vocabulary Semantic Segmentation

  2. One-Shot Calibration via Model Editing

    Nov 16, 2026Jiaxing Qiu, Luke Hakso, Roxana Daneshjou +2Model EditingLanguage Model Editing

  3. CPNav: Calibrated Vision-Language Navigation for Object-Goal Search in Unseen Environments

    Nov 7, 2026Qizhao Chen, Yuanhong Zeng, Shoh Nishino +1Vision-Language NavigationObject-Goal Navigation

  4. Density-Aware Calibration for Resource-Efficient Medical Vision-Language Models

    Oct 31, 2026Tien-Dat Dam, Huu-Thuan Huynh, Minh-Triet Tran +1VLM CalibrationMedical VLMs

  5. Beyond CLIP: Calibrated (Few-Shot) Defect Detection With System One Decision Models

    Oct 30, 2026Fadel M. Megahed, Ying-Ju Chensystem one decision modelsPrompt Learning for VLMs

  6. Benchmarking color judgment in multimodal LLMs with CIE-computable ground truth

    Oct 29, 2026Shuwei Yue, Kaida Xiao, Zhihua Wang +1MLLM EvaluationVLM Calibration

  7. M-PFT: Angular Margin for Calibrated Prompt-Tuning of Fine-Tuned Vision-Language Models

    Oct 29, 2026Shihab Aaqil Ahamed, Mohamed Ansaff, Nang Khin Myat Lay Phyu +4VLM CalibrationPrompt Learning for VLMs

  8. Surrogate Uncertainty for Black-box Visual Grounding

    Oct 29, 2026Hanbo Zhang, Tianrun Hu, Cedar Site Bai +2Uncertainty QuantificationVisual Grounding

  9. PathJev: Decision-Native Vision-Language Modelling for Histopathology

    Oct 23, 2026Cristian Tommasino, Antonio M. RinaldiComputational PathologyHistopathology Image Classification

  10. Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation

    Oct 22, 2026Dominik Engel, Yousef Rajeh, Leon Sick +1Open-Vocabulary 3D SegmentationVLM Calibration

  11. UniJev: Separating Accuracy from Calibration in Trimodal System One Decision Models

    Oct 19, 2026Jianing Wang, Youwen Jian, Derong Wang +5Model CalibrationVLM Calibration

  12. How Murky Before a VLM Knows It Cannot See? Accuracy, Confidence and Abstention on a Physical Visibility Axis

    Oct 16, 2026Mochamad Rizal Fauzan, Irgi SuryaVLM EvaluationVLM Robustness

  13. DICL-Qwen3VL: Decoupled Confidence Calibration for Interpretable Workwear Detection

    Oct 16, 2026Meng Wang, Wu Zeng, Mei Li +1VLM CalibrationVLM Interpretability

  14. Localized Conformal Safety Monitoring with Vision-Language Models for Autonomous Driving

    Oct 2, 2026Luís Marques, Rong Fang, Disha Kamale +1Autonomous Driving Safety EvaluationAutonomous Driving

  15. Towards Reliable Vision-Language Models for Autonomous Driving

    Oct 1, 2026Manasa Mariam Mammen, Priyanka Mary Mammen, Zafer Kayatas +1VLM RobustnessImage Corruption Robustness

  16. SubRot: Signed Gradient Subspace Calibration for VLM Rotation Quantization

    Sep 28, 2026Zhenhao Shang, Haizhao Jing, Haokui Zhang +4VLM QuantizationRotation-Based Quantization

  17. Domain Recentering and Confidence-Weighted Prior Calibration for Vision-Language Models

    Sep 24, 2026Youngeun Seol, Jimin Shin, Heeseo Yoon +1Source-Free Domain AdaptationUnsupervised Domain Adaptation

  18. Calibrated Probabilistic Obstruction Reasoning with Vision-Language Models for Grasping in Clutter

    Sep 16, 2026Thanh-Tuan Tran, Ngoc-Chien Chu, Thanh Nguyen Canh +3Robotic GraspingRobotic Manipulation

  19. The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models

    Sep 16, 2026Jisoo Yang, Jaeho Han, Trung X. Pham +1VLM EvaluationConfidence Estimation in Language Models

  20. Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery?

    Sep 15, 2026Seung Jae Lieu, Diego Morra, Chiara Cadoni +3VLM EvaluationUrban Planning

  21. Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuning

    Sep 15, 2026Yuwei Liang, Jian Liang, Dapeng Hu +2Post-Hoc CalibrationTest-Time Adaptation of VLMs

  22. Human-Grounded Calibration for Long-Text Image-Text Congruence in Vision-Language Models

    Sep 14, 2026Alessandro Gambetti, Qiwei HanText-Image AlignmentVision-Language Retrieval

  23. Target-Checked Reliability Score Refinement for Video Question Answering

    Sep 9, 2026Guoxiang Ren, Rohitash ChandraConfidence Estimation for VLMsVideo Question Answering