Vision-Language Pretraining

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 50

All topics

What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?
Oct 7, 2026Nikos Giakoumoglou, Paschalis Giakoumoglou, Andreas Floros +2Vision-Language Pretraining Contrastive Learning
Imperial College London, London, UK · Centre for Research and Technology Hellas, Thessaloniki, Greece
VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training
Sep 29, 2026Yuanwei Hu, Bo Peng, Yuheng Jia +3Clustering Deep Clustering
University College London · The University of Queensland · Southeast University +2
SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis
Sep 28, 2026Hangyul Yoon, Hyungyung Lee, Edward Choi +1Contrastive Learning Radiology VLMs
Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST)
Clinical Trajectory Alignment for Medical Vision-Language Pre-training
Sep 28, 2026Huimin Yan, Xian Yang, Zhi Wang +1Multimodal Pretraining Longitudinal Medical Image Analysis
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, China · Alliance Manchester Business School, The University of Manchester, Manchester, UK
Med-AR: Autoregressive Vision-Language Pretraining for Long-Tailed Chest X-Ray Classification and Uncertainty-Aware Evaluation
Sep 24, 2026Janhavi Prabhu, Sahil, Akshay V +3Long-Tailed Classification Chest X-Ray Classification
Qure.ai, Bangalore, India
TDDN: Text-aligned Diffused DINO Network for Puzzle Understanding
Sep 7, 2026Harsha Patnala, Debopriyo Banerjee, Ayush Sunil Munot +1Visual Reasoning Vision-Language Alignment
Eightfold AI · Inception42 · Indian Institute of Technology Kharagpur
On the Design Fundamentals of Pixel Text Representation Learning
Sep 1, 2026Chaohao Yuan, Ruifeng Yuan, Zhuoxu Huang +4Visual Representation Learning Vision-Language Alignment
1The Chinese University of Hong Kong · 2DAMO Academy, Alibaba Group · 3Fudan University +4
CL4D: Contrastive Language-4D Pretraining for Vision-Language Reasoning in Dynamic Scenes
Aug 19, 2026Kumal Hewagamage, Isuranga Senavirathne, Sasika Amarasinghe +4Vision-Language Models Vision-Language Alignment
University of Moratuwa, Sri Lanka · Singapore-MIT Alliance for Research & Technology (SMART) Centre, Singapore · Agency for Science, Technology and Research (A*STAR), Singapore
A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports
Jul 30, 2026Jia Yu, Yan Zhu, Yili He +12Endoscopy Medical VLMs
Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai, China. · Shanghai Collaborative Innovation Center of Endoscopy, Shanghai, China. · Shanghai Institute for Advanced Study of Zhejiang University, Shanghai, China. +5
DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes
Jul 27, 2026Jiahao Xie, Zhongbin Guo, Qianle Wang +4Training Data Curation Data Mixture Optimization
ByteDance · ByteDance Commercial AI Team
UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
Jul 25, 2026Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati +4Vision-Language Models Efficient VLM Inference
Samsung AI Cambridge · Technical University of Iasi · Queen Mary University of London
AspectCLIP: Optimizing CLIP Representation Space via Aspect-Guided Consistency Regularization
Jul 15, 2026Yiyang Yao, Shanglin Liu, Jianming Lv +4Vision-Language Alignment Multimodal Contrastive Learning
School of Computer Science and Technology, South China University of Technology, Guangzhou, China
MonkeyOCRv2: A Visual-Text Foundation Model for Document AI
Jul 13, 2026Yuliang Liu, Zhang Li, Ziyang Zhang +11Document Parsing Multimodal Foundation Models
1Huazhong University of Science and Technology · 2Kingsoft Office
SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception
Jul 13, 2026Mingjie Xie, Guangjun He, Dongli Xu +5VLM Robustness Vision-Language Grounding
1Beihang University · 2State Key Laboratory of Space Information System and Integrated Application · 3Independent Researcher +3
LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives
Jul 1, 2026Lukas Kuhn, Giuseppe Serra, Randall Balestriero +1Visual Representation Learning Vision-Language Pretraining
German Cancer Research Center · German Cancer Consortium · Brown University +1
SonoCLIP: Mask-Guided Region-Aware Vision-Language Pretraining for Fetal Ultrasound Analysis
Jun 28, 2026Hang Su, Chao Sun, Zhaofan Li +3Medical Imaging Foundation Models Ultrasound Imaging
School of Computer Science, Wuhan University, Wuhan, China · Institute of Artificial Intelligence, Wuhan University, Wuhan, China · Department of Ultrasound, Renmin Hospital of Wuhan University, Wuhan, China +2
DataComp-VLM: Improved Open Datasets for Vision-Language Models
Jun 26, 2026Matteo Farina, Vishaal Udandarao, Thao Nguyen +33VLM Evaluation Training Data Curation
University of Trento · 2Tübingen AI Center, University of Tübingen · University of Cambridge +12
ReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP
Jun 25, 2026Sicheng Zhang, Muzammal Naseer, Binzhu Xie +5Compositional Reasoning Multimodal Reasoning
Khalifa University · University of Western Australia · The Chinese University of Hong Kong +2
TuringViT: Making SOTA Vision Transformers Accessible to All
Jun 23, 2026Qiman Wu, Hanlin Chen, Lyujie Chen +19Vision Foundation Models Vision Transformer
Foundation Model Team, Xpeng Inc.
Data Selection Through Iterative Self-Filtering for Vision-Language Settings
Jun 22, 2026Andrei Liviu Nicolicioiu, Sarvjeet Singh Ghotra, Morgane M. Moss +1Vision-Language Models Training Data Curation
Mila, Université de Montréal
Beyond Native Success: Auditing Deployment-Interface Exposure of CLIP Backdoors
Jun 16, 2026Kunlan Xiang, Haomiao Yang, Wenbo JiangVLM Robustness Adversarial Attacks on VLMs
University of Electronic Science and Technology of China
ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining
Jun 15, 2026Hao Li, Ganlong Zhao, Yufei Liu +8Cross-Embodiment Robot Learning Vision-Language-Action Models
ACE Robotics · CUHK MMLab · SJTU +2
Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality
Jun 11, 2026Wei Li, Zhen Huang, Xinmei TianCross-Modal Learning Vision-Language Models
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China · Independent Researcher
X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining
Jun 7, 2026Miracle Kang, Lights Shi, Lucy Liang +10Vision-Language-Action Models Vision-Language Pretraining
X SQUARE ROBOT · City University of Hong Kong · Tsinghua University
WALL-WM: Carving World Action Modeling at the Event Joints
Jun 1, 2026Shalfun Li, Victor Yao, Charles Yang +28World Model Learning Vision-Language-Action Models
X Square Robot Team
"Înţelegi Româneşte?'' A Recipe for Romanian Vision-Language Models
May 29, 2026Mihai Masala, Marius Leordeanu, Mihai Dascalu +1VLM Evaluation Vision-Language Models
National University of Science and Technology POLITEHNICA Bucharest, Romania
Wall-OSS-0.5 Technical Report
May 29, 2026Ryan Yu, Pushi Zhang, Starrick Liu +24Zero-Shot Robotic Manipulation Cross-Embodiment Robot Learning
X Square Robot Team
From Pixels to Words -- Towards Native One-Vision Models at Scale
May 27, 2026Haiwen Diao, Jiahao Wang, Penghao Wu +18Vision-Language Models Video-Language Models
S-Lab, NTU · SenseTime Research · 3DLUT
GEM: Generative Supervision Helps Embodied Intelligence
May 27, 2026Ruowen Zhao, Bangguo Li, Zuyan Liu +9Vision-Language-Action Models Embodied AI
Tsinghua University · Tencent Hunyuan
Rethinking VLM Representation for VLA Initialization
May 25, 2026Weifeng Lin, Siyuan Huang, Hao Li +5VLM Adaptation Low-Rank Adaptation
1CUHK · 4ACE Robotics · 3Peking University +1

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?

Nikos Giakoumoglou, Paschalis Giakoumoglou, Andreas Floros, Kleanthis Marios Papadopoulos+1

Imperial College London, London, UK · Centre for Research and Technology Hellas, Thessaloniki, Greece

Vision-Language Pretraining Contrastive Learning Synthetic Data Generation

Open paper AskPDF arXiv

Synthetic hard negatives generated in the representation space have proven effective for unimodal self-supervised learning, but transferring this idea to vision-language pretraining is not straightforward. We analyze six representation-space synthesis strategies and identify two failure modes in their transfer to vision-language pretraining: cross-modal constructions that produce overly easy negatives or pull them toward the query, and intra-modal constructions that incorporate the matched positive. We also observe logit-scale saturation when training with synthetic hard negatives and a learnable temperature, and find that fixing the temperature improves downstream performance. Using this geometric analysis we propose SNAP, which generates intra-modal hard negatives that never involve the positive from either modality, avoiding both failure modes entirely. SNAP is model-agnostic, requires no external generative models, and adds less than 10% training time overhead. Evaluated on top of CLIP and FLIP across multiple architectures and datasets, SNAP delivers consistent improvements on zero-shot retrieval, zero-shot classification, and linear probe evaluation.

Momentum

Related topics

Latest papers 50