cs.CVOct 31, 2026

Density-Aware Calibration for Resource-Efficient Medical Vision-Language Models

Authors: Tien-Dat Dam, Huu-Thuan Huynh, Minh-Triet Tran, Trung-Nghia Le

Organizations: University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam

Abstract

Medical vision-language models (VLMs) show promise for clinical image understanding, but their computational and memory requirements hinder deployment on resource-constrained clinical hardware. Post-training quantization (PTQ) offers a practical compression strategy without retraining, yet its effectiveness depends strongly on the calibration data. We propose CoreCalib, a training-free and quantizer-agnostic framework for constructing compact calibration sets that preserve the target medical VQA distribution. CoreCalib stratifies data by clinical question class, identifies representative visual modes using frozen DINOv2 features with UMAP and HDBSCAN, and expands selected images into class-balanced question-answer pairs. On Kvasir-VQA-x1, we evaluate CoreCalib with MASQuant, GPTQ, and QIG using a fixed 128-pair budget. CoreCalib achieves the best METEOR with all three quantizers and the best ROUGE-L with MASQuant and GPTQ. With W4A8 MASQuant, it improves METEOR from 0.5672 with COCO calibration to 0.6234, recovering approximately 41% of the degradation relative to BF16. These results support CoreCalib as a data-centric component for resource-efficient medical VLM deployment. Project page: https://magnusdtd.github.io/CoreCalib/ .

Figures & tables

Explore similar work

CardsList
  1. MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

    Aug 27, 2026Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan +6VLM EvaluationVLM Robustness

  2. Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation

    Apr 2, 2026Ji Young Byun, Young-Jin Park, Jean-Philippe Corbeil +1Vision-Language ModelsVLM Hallucination