Contrastive Learning for VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

17 papers4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoders
Oct 8, 2026Tao Yang, Jianying ZhouContrastive Learning for VLMs Adversarial Attacks on VLMs
Singapore University of Technology and Design
Localization Lens for Improving Medical Vision-Language Models
Oct 3, 2026Hasan Farooq, Murtaza Taj, Mehwish Nasim +1Contrastive Learning for VLMs Medical VLMs
Lahore University of Management Sciences, Lahore, Pakistan · The University of Western Australia, Perth, Australia · Information Technology University of the Punjab, Lahore, Pakistan
Targeted Visual Counterfactual Explanations for Contrastive Vision-Language Model
Sep 29, 2026Van Bach Nguyen, Jörg Schlötterer, Christin SeiferVision-Language Models VLM Interpretability
Marburg University, Germany
One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models
Sep 28, 2026Aditya Sharma, Divya SaxenaCross-Modal Retrieval Modality Gap in VLMs
Indian Institute of Technology Jodhpur
Re-calibrated Contrastive Loss for Transformation-Aware Prompt Conditioning in Vision-Language Models
Sep 7, 2026Seungmin Oh, Seunghun Kang, Jongbin RyuVLM Adaptation Contrastive Learning for VLMs
Department of Artificial Intelligence, Ajou University, Suwon, South Korea · Department of Computer Engineering, Ajou University, Suwon, South Korea
Hallucination Mitigation for Large Vision-Language Models via Implicit Feature Stabilization
Aug 30, 2026Aditi Sarker, Rafi Ibn Sultan, Hui Zhu +2VLM Robustness Vision-Language Models
Department of Computer Science, Wayne State University, MI, USA · Institute for AI and Data Science (AIDaS), Wayne State University, MI, USA
MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
Aug 26, 2026Jongsuk Kim, Qiyu Wu, Zhuoyuan Mao +3Contrastive Learning for VLMs Vision-Language Models
KAIST · Sony Group Corporation
Text as Partial Constraint: Core-Residual Alignment for Robust Vision-Language Learning
Jul 3, 2026Chengzhen Yu, Canran Xiao, Siyuan Ma +1VLM Robustness Contrastive Learning
1Nanyang Technological University (NTU) · 2Shenzhen Campus of Sun Yat-sen University
Is CLIP Cross-Eyed? Revealing and Mitigating Center Bias in the CLIP Family
Apr 7, 2026Oscar Chew, Hsiao-Ying Huang, Kunal Jain +3Contrastive Learning for VLMs Vision-Language Models
Texas A&M University · National Taiwan University · ASUS +1
CoME-VL: Scaling Complementary Multi-Encoder Vision-Language Learning
Apr 3, 2026Ankan Deria, Komal Kumar, Xilin He +4Vision-Language Models Multimodal Fusion
Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, UAE
No Hard Negatives Required: Concept Centric Learning Leads to Compositionality without Degrading Zero-shot Capabilities of Contrastive Models
Mar 26, 2026Hai X. Pham, David T. Hoffmann, Ricardo Guerrero +1Vision-Language Models Compositional Reasoning in VLMs
Samsung AI Center Cambridge, UK
Complementary Text-Guided Attention for Zero-Shot Adversarial Robustness
Mar 19, 2026Lu Yu, Haiyang Zhang, Changsheng XuAttention-Based Adversarial Attacks Contrastive Learning for VLMs
School of Computer Science and Engineering, Tianjin University of Technology, 300384, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China · School of Artificial Intelligence, University of the Chinese Academy of Sciences, Beijing, 100049, China
AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification
Mar 8, 2026Hamza Mooraj, George Pantazopoulos, Alessandro SugliaContrastive Learning for VLMs Plant Disease Classification
School of Mathematical and Computer Sciences Heriot-Watt University · Institute for Language, Cognition and Computation, School of Informatics University of Edinburgh
Half-Truths Break Similarity-Based Retrieval
Feb 27, 2026Bora Kargi, Arnas Uselis, Seong Joon OhContrastive Learning for VLMs Vision-Language Models
University of Tübingen, Tübingen AI Center · Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA)
CLIMP: Contrastive Language-Image Mamba Pretraining
Jan 11, 2026Nimrod Shabtay, Itamar Zimerman, Eli Schwartz +1Vision-Language Pretraining VLM Robustness
Tel-Aviv University · IBM Research
Contrastive vision-language learning with paraphrasing and negation
Nov 20, 2025Kwun Ho Ngan, Saman Sadeghi Afgeh, Joe Townsend +1VLM Robustness VLM Reasoning
Fujitsu Research of Europe · United Kingdom · City St George’s, University of London
Align Where the Words Look: Cross-Attention-Guided Patch Alignment with Contrastive and Transport Regularization for Bengali Captioning
Sep 22, 2025Riad Ahmed Anonto, Sardar Md. Saffat Zabin, M. Saifur RahmanImage Captioning VLM Adaptation
Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh

Oct 8, 2026J/K move · Enter open · S save

Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoders

Tao Yang, Jianying Zhou

Singapore University of Technology and Design

Contrastive Learning for VLMs Adversarial Attacks on VLMs Vision-Language Models VLM Evaluation VLM Robustness

Open paper AskPDF arXiv

Adversarial attacks on vision-language models optimize an image toward a text target, then cite the attacked model's similarity score as evidence of success. We ask whether that score - victim-space target alignment (VTS) - predicts recovery of the target by an independent model. We first build a measurement instrument: supervised judges outside the attacked geometry, real-target blend controls, shuffled-target negatives, and a reference level derived from a 50% target-image blend. Two preregistered studies then compare six contrastive encoders under a matched attack at three perturbation budgets. Robustly trained encoders (FARE, TeCoA, PMG, TRADES) transfer substantially more independent evidence than vanilla CLIP or SigLIP; all eight contrasts reject at the bootstrap floor. However, no cell reaches the blend-derived reference level. The three best cells fall within its replication band, leaving practical recovery undecided. Within robust encoders, per-sample alignment gain correlates with evidence gain (

ρ= 0.24-0.51

); within vanilla CLIP the correlation is consistent with zero. Across encoders we find no monotone alignment-evidence relation. VTS is therefore informative only within a fixed robust encoder, and we provide a reporting protocol in its place.