cs.CVNov 13, 2026

CARET: Clinical Assertion-Aware Residual Calibration for Medical Image-Text Alignment

Authors: Yifan Jin, Tao Liu

Organizations: Sun Yat-sen University, Guangzhou, Guangdong, China

Abstract

Medical vision--language models align radiographs with free-text reports, but radiology language distinguishes explicit findings, negation, uncertainty, and omission. Treating these report states as interchangeable during feature adaptation can change image predictions without supporting visual evidence. We introduce CARET, a parameter-efficient image-side adaptation method that retains pretrained image and text encoders and uses paired positive and negative clinical statements to define a low-dimensional semantic correction space. A bounded residual adjusts image features along these directions, while assertion-aware supervision distinguishes explicitly present and absent findings from uncertain or unmentioned ones. At inference, the adapter requires only an image; textual references remain fixed. In zero-shot classification, CARET achieves 67.80% accuracy on CheXpert 5×\times200 and 65.20% on RSNA. These scores exceed the published CLEFT-LoRA results by 1.06 and 0.27 percentage points, respectively. Comparisons to literature baselines are informative but not controlled for differences in pretraining and evaluation protocols. Ablations across eight components identify the four-state distinction and assertion objective as major contributors, with task-dependent effects for nonmention preservation and pair alignment.

Figures & tables

Explore similar work

CardsList
  1. Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays

    Sep 17, 2025Hanbin Ko, Gihun Cho, Inhyeok Baek +5Image-Text RetrievalRadiology VLMs

  2. Vision-language models for chest radiography do not always need the image

    Jun 16, 2026Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams +4VLM EvaluationMedical Visual Question Answering

  3. Findings-aware Language-Image Pretraining for CT Volume Interpretation

    Nov 2, 2026Takuro Shimaya, Junya Inoh, Masahiro SaikoMedical VLMsMedical Image Representation Learning