Contrastive Learning for VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

26 papers5 papers in the last four weeks, up 25% on the four weeks before. 0.0% of all new papers.

DreamRealm: Exploring and Materializing Hallucination Worlds for Trustworthy Vision-Language Models
Nov 4, 2026Benlei Cui, Chunyang Chai, Yiliang Zhang +6VLM Hallucination VLM Hallucination Mitigation
Yuvion Team, Alibaba Group
Discarded or Unreachable? Diagnosing and Repairing Spatial Relation Failure in Vision--Language Similarity
Nov 3, 2026Mahdi Mostajabdaveh, Zirui ZhouVision-Language Grounding Representation Geometry in VLMs
Huawei Technologies Canada
Latent Visual Contrastive Reasoning with Multi-Branch Exploration
Oct 27, 2026Kegeng Tang, Jingbo Wang, Yangming Zhang +3MLLM Reasoning Latent Visual Reasoning
Department of Computer Science & Engineering, University of Tennessee at Chattanooga, Chattanooga, TN, USA · Department of Computer Science, University of Texas at Arlington, Arlington, TX, USA · Department of Electrical Engineering, The Pennsylvania State University, University Park, PA, USA
Recursive CLIP: Vision-Language Distillation with Looped Transformer Blocks
Oct 25, 2026Szymon Mazurek, Alberto Presta, Michał ByraVision-Language Models Recurrent Transformers
Samsung AI Center Warsaw, Poland · AGH University of Krakow, Poland · ACC Cyfronet AGH, Poland +1
Meta-CLIP: Metadata-Conditioned Contrastive Learning for Zero-Shot Disease Inference
Oct 24, 2026Haozhe Luo, Nirosh Sivanesan, Shelley Zixin Shu +2Medical VLMs Contrastive Learning for VLMs
University of Bern, Bern, Switzerland · ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
Separating Angular Alignment from Radial Ordering in Hyperbolic Vision-Language Models
Oct 23, 2026Mingyang Gao, Yuwei Wu, Shiyu Luo +8Vision-Language Models Hyperbolic Representation Learning
Beijing Institute of Technology · Shenzhen MSU-BIT University
Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoders
Oct 8, 2026Tao Yang, Jianying ZhouContrastive Learning for VLMs Adversarial Attacks on VLMs
Singapore University of Technology and Design
Localization Lens for Improving Medical Vision-Language Models
Oct 3, 2026Hasan Farooq, Murtaza Taj, Mehwish Nasim +1Contrastive Learning for VLMs Medical VLMs
Lahore University of Management Sciences, Lahore, Pakistan · The University of Western Australia, Perth, Australia · Information Technology University of the Punjab, Lahore, Pakistan
Targeted Visual Counterfactual Explanations for Contrastive Vision-Language Model
Sep 29, 2026Van Bach Nguyen, Jörg Schlötterer, Christin SeiferVision-Language Models VLM Interpretability
Marburg University, Germany
One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models
Sep 28, 2026Aditya Sharma, Divya SaxenaCross-Modal Retrieval Modality Gap in VLMs
Indian Institute of Technology Jodhpur
Re-calibrated Contrastive Loss for Transformation-Aware Prompt Conditioning in Vision-Language Models
Sep 7, 2026Seungmin Oh, Seunghun Kang, Jongbin RyuVLM Adaptation Contrastive Learning for VLMs
Department of Artificial Intelligence, Ajou University, Suwon, South Korea · Department of Computer Engineering, Ajou University, Suwon, South Korea
Hallucination Mitigation for Large Vision-Language Models via Implicit Feature Stabilization
Aug 30, 2026Aditi Sarker, Rafi Ibn Sultan, Hui Zhu +2VLM Robustness Vision-Language Models
Department of Computer Science, Wayne State University, MI, USA · Institute for AI and Data Science (AIDaS), Wayne State University, MI, USA
MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
Aug 26, 2026Jongsuk Kim, Qiyu Wu, Zhuoyuan Mao +3Contrastive Learning for VLMs Vision-Language Models
KAIST · Sony Group Corporation
Text as Partial Constraint: Core-Residual Alignment for Robust Vision-Language Learning
Jul 3, 2026Chengzhen Yu, Canran Xiao, Siyuan Ma +1VLM Robustness Contrastive Learning
1Nanyang Technological University (NTU) · 2Shenzhen Campus of Sun Yat-sen University
Is CLIP Cross-Eyed? Revealing and Mitigating Center Bias in the CLIP Family
Apr 7, 2026Oscar Chew, Hsiao-Ying Huang, Kunal Jain +3Contrastive Learning for VLMs Vision-Language Models
Texas A&M University · National Taiwan University · ASUS +1
CoME-VL: Scaling Complementary Multi-Encoder Vision-Language Learning
Apr 3, 2026Ankan Deria, Komal Kumar, Xilin He +4Vision-Language Models Multimodal Fusion
Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, UAE
No Hard Negatives Required: Concept Centric Learning Leads to Compositionality without Degrading Zero-shot Capabilities of Contrastive Models
Mar 26, 2026Hai X. Pham, David T. Hoffmann, Ricardo Guerrero +1Vision-Language Models Compositional Reasoning in VLMs
Samsung AI Center Cambridge, UK
Complementary Text-Guided Attention for Zero-Shot Adversarial Robustness
Mar 19, 2026Lu Yu, Haiyang Zhang, Changsheng XuAttention-Based Adversarial Attacks Contrastive Learning for VLMs
School of Computer Science and Engineering, Tianjin University of Technology, 300384, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China · School of Artificial Intelligence, University of the Chinese Academy of Sciences, Beijing, 100049, China
AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification
Mar 8, 2026Hamza Mooraj, George Pantazopoulos, Alessandro SugliaContrastive Learning for VLMs Plant Disease Classification
School of Mathematical and Computer Sciences Heriot-Watt University · Institute for Language, Cognition and Computation, School of Informatics University of Edinburgh
Half-Truths Break Similarity-Based Retrieval
Feb 27, 2026Bora Kargi, Arnas Uselis, Seong Joon OhContrastive Learning for VLMs Vision-Language Models
University of Tübingen, Tübingen AI Center · Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA)
Tune-Up Open-Weight CLIP: Optimization Framework for Self-Supervised Fine-tuning of CLIP
Jan 14, 2026Anant Mehta, Xingyu Chen, Xiyuan Wei +1VLM Adaptation Contrastive Language-Image Pretraining
Department of Computer Science & Engineering, Texas A&M University, USA
CLIMP: Contrastive Language-Image Mamba Pretraining
Jan 11, 2026Nimrod Shabtay, Itamar Zimerman, Eli Schwartz +1Vision-Language Pretraining VLM Robustness
Tel-Aviv University · IBM Research
Contrastive vision-language learning with paraphrasing and negation
Nov 20, 2025Kwun Ho Ngan, Saman Sadeghi Afgeh, Joe Townsend +1Vision-Language Retrieval Vision-Language Alignment
Fujitsu Research of Europe · United Kingdom · City St George’s, University of London
ProCLIP: Progressive Vision-Language Alignment via LLM-based Embedder
Oct 21, 2025Xiaoxing Hu, Kaicheng Yang, Ziqi Ye +7Vision-Language Alignment VLM Adaptation
SAIS & SCS, Shanghai Jiao Tong University · Shanghai Innovation Institute · DeepGlint +3
Cattle-CLIP: A Multimodal Framework for Dairy Cattle Behaviour Recognition from Video
Oct 10, 2025Huimin Liu, Jing Gao, Daria Baran +3Animal Behavior Analysis Video-Language Models
Bristol Veterinary School, University of Bristol, Langford House, Dolberry, Churchill, Bristol, BS40 5DU, United Kingdom · School of Computer Science, University of Bristol, Merchant Venturers Building, Woodland Road, Bristol, BS8 1UB, United Kingdom
Align Where the Words Look: Cross-Attention-Guided Patch Alignment with Contrastive and Transport Regularization for Bengali Captioning
Sep 22, 2025Riad Ahmed Anonto, Sardar Md. Saffat Zabin, M. Saifur RahmanImage Captioning VLM Adaptation
Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh

Nov 4, 2026J/K move · Enter open · S save

DreamRealm: Exploring and Materializing Hallucination Worlds for Trustworthy Vision-Language Models

Benlei Cui, Chunyang Chai, Yiliang Zhang, Xiaoqian Xia+5

Yuvion Team, Alibaba Group

VLM Hallucination VLM Hallucination Mitigation Visual Evidence Sensitivity in VLMs Synthetic Data Generation for VLMs Contrastive Learning for VLMs

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) remain prone to visual hallucinations when correct responses depend on subtle visual details. One line of post-training methods constructs visual contrasts through image retrieval, generation, or editing to improve fine-grained perception. However, these contrasts are not built around the model's own hallucinations, making it difficult to ensure that supervision targets its actual perceptual confusions. Another line suppresses hallucinated outputs through response correction or preference learning under fixed image conditions, but leaves the design of supervision signals on the image-input side underexplored. Inspired by how humans correct misconceptions, we introduce DreamRealm, a post-training framework that explores diverse latent hallucinated responses, minimally edits original images to make them valid, and trains the model to assign each response to its supporting image. Our data construction pipeline, RealmForgePipe, mines observed and latent hallucinations from a target model and materializes them through image editing. Using hallucinations mined from Qwen3.5-9B, we construct the Realized Hallucination Dataset (RealmSet). Our training objective, RealmMatch, constructs a likelihood score matrix across image--response combinations and combines supervised learning with bidirectional contrastive losses to reinforce correct matches and suppress mismatches. Compared with the original Qwen3.5-9B model, DreamRealm improves performance across six visual perception and hallucination benchmarks, including gains of 7.26 percentage points in HallusionBench accuracy and 11.34 percentage points in MMVP pair accuracy. Together with ablation studies on data construction and training methods, these results demonstrate that hallucination assignment is a promising paradigm for improving visual perception and mitigating hallucinations.