Vision-Language Pretraining

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 50

All topics

What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?
Oct 7, 2026Nikos Giakoumoglou, Paschalis Giakoumoglou, Andreas Floros +2Vision-Language Pretraining Contrastive Learning
Imperial College London, London, UK · Centre for Research and Technology Hellas, Thessaloniki, Greece
VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training
Sep 29, 2026Yuanwei Hu, Bo Peng, Yuheng Jia +3Clustering Deep Clustering
University College London · The University of Queensland · Southeast University +2
SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis
Sep 28, 2026Hangyul Yoon, Hyungyung Lee, Edward Choi +1Contrastive Learning Radiology VLMs
Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST)
Clinical Trajectory Alignment for Medical Vision-Language Pre-training
Sep 28, 2026Huimin Yan, Xian Yang, Zhi Wang +1Multimodal Pretraining Longitudinal Medical Image Analysis
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, China · Alliance Manchester Business School, The University of Manchester, Manchester, UK
Med-AR: Autoregressive Vision-Language Pretraining for Long-Tailed Chest X-Ray Classification and Uncertainty-Aware Evaluation
Sep 24, 2026Janhavi Prabhu, Sahil, Akshay V +3Long-Tailed Classification Chest X-Ray Classification
Qure.ai, Bangalore, India
TDDN: Text-aligned Diffused DINO Network for Puzzle Understanding
Sep 7, 2026Harsha Patnala, Debopriyo Banerjee, Ayush Sunil Munot +1Visual Reasoning Vision-Language Alignment
Eightfold AI · Inception42 · Indian Institute of Technology Kharagpur
On the Design Fundamentals of Pixel Text Representation Learning
Sep 1, 2026Chaohao Yuan, Ruifeng Yuan, Zhuoxu Huang +4Visual Representation Learning Vision-Language Alignment
1The Chinese University of Hong Kong · 2DAMO Academy, Alibaba Group · 3Fudan University +4
CL4D: Contrastive Language-4D Pretraining for Vision-Language Reasoning in Dynamic Scenes
Aug 19, 2026Kumal Hewagamage, Isuranga Senavirathne, Sasika Amarasinghe +4Vision-Language Models Vision-Language Alignment
University of Moratuwa, Sri Lanka · Singapore-MIT Alliance for Research & Technology (SMART) Centre, Singapore · Agency for Science, Technology and Research (A*STAR), Singapore
A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports
Jul 30, 2026Jia Yu, Yan Zhu, Yili He +12Endoscopy Medical VLMs
Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai, China. · Shanghai Collaborative Innovation Center of Endoscopy, Shanghai, China. · Shanghai Institute for Advanced Study of Zhejiang University, Shanghai, China. +5
DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes
Jul 27, 2026Jiahao Xie, Zhongbin Guo, Qianle Wang +4Training Data Curation Data Mixture Optimization
ByteDance · ByteDance Commercial AI Team
UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
Jul 25, 2026Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati +4Vision-Language Models Efficient VLM Inference
Samsung AI Cambridge · Technical University of Iasi · Queen Mary University of London
AspectCLIP: Optimizing CLIP Representation Space via Aspect-Guided Consistency Regularization
Jul 15, 2026Yiyang Yao, Shanglin Liu, Jianming Lv +4Vision-Language Alignment Multimodal Contrastive Learning
School of Computer Science and Technology, South China University of Technology, Guangzhou, China
MonkeyOCRv2: A Visual-Text Foundation Model for Document AI
Jul 13, 2026Yuliang Liu, Zhang Li, Ziyang Zhang +11Document Parsing Multimodal Foundation Models
1Huazhong University of Science and Technology · 2Kingsoft Office
SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception
Jul 13, 2026Mingjie Xie, Guangjun He, Dongli Xu +5VLM Robustness Vision-Language Grounding
1Beihang University · 2State Key Laboratory of Space Information System and Integrated Application · 3Independent Researcher +3
LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives
Jul 1, 2026Lukas Kuhn, Giuseppe Serra, Randall Balestriero +1Visual Representation Learning Vision-Language Pretraining
German Cancer Research Center · German Cancer Consortium · Brown University +1
SonoCLIP: Mask-Guided Region-Aware Vision-Language Pretraining for Fetal Ultrasound Analysis
Jun 28, 2026Hang Su, Chao Sun, Zhaofan Li +3Medical Imaging Foundation Models Ultrasound Imaging
School of Computer Science, Wuhan University, Wuhan, China · Institute of Artificial Intelligence, Wuhan University, Wuhan, China · Department of Ultrasound, Renmin Hospital of Wuhan University, Wuhan, China +2
DataComp-VLM: Improved Open Datasets for Vision-Language Models
Jun 26, 2026Matteo Farina, Vishaal Udandarao, Thao Nguyen +33VLM Evaluation Training Data Curation
University of Trento · 2Tübingen AI Center, University of Tübingen · University of Cambridge +12
ReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP
Jun 25, 2026Sicheng Zhang, Muzammal Naseer, Binzhu Xie +5Compositional Reasoning Multimodal Reasoning
Khalifa University · University of Western Australia · The Chinese University of Hong Kong +2
TuringViT: Making SOTA Vision Transformers Accessible to All
Jun 23, 2026Qiman Wu, Hanlin Chen, Lyujie Chen +19Vision Foundation Models Vision Transformer
Foundation Model Team, Xpeng Inc.
Data Selection Through Iterative Self-Filtering for Vision-Language Settings
Jun 22, 2026Andrei Liviu Nicolicioiu, Sarvjeet Singh Ghotra, Morgane M. Moss +1Vision-Language Models Training Data Curation
Mila, Université de Montréal
Beyond Native Success: Auditing Deployment-Interface Exposure of CLIP Backdoors
Jun 16, 2026Kunlan Xiang, Haomiao Yang, Wenbo JiangVLM Robustness Adversarial Attacks on VLMs
University of Electronic Science and Technology of China
ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining
Jun 15, 2026Hao Li, Ganlong Zhao, Yufei Liu +8Cross-Embodiment Robot Learning Vision-Language-Action Models
ACE Robotics · CUHK MMLab · SJTU +2
Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality
Jun 11, 2026Wei Li, Zhen Huang, Xinmei TianCross-Modal Learning Vision-Language Models
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China · Independent Researcher
X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining
Jun 7, 2026Miracle Kang, Lights Shi, Lucy Liang +10Vision-Language-Action Models Vision-Language Pretraining
X SQUARE ROBOT · City University of Hong Kong · Tsinghua University
WALL-WM: Carving World Action Modeling at the Event Joints
Jun 1, 2026Shalfun Li, Victor Yao, Charles Yang +28World Model Learning Vision-Language-Action Models
X Square Robot Team
"Înţelegi Româneşte?'' A Recipe for Romanian Vision-Language Models
May 29, 2026Mihai Masala, Marius Leordeanu, Mihai Dascalu +1VLM Evaluation Vision-Language Models
National University of Science and Technology POLITEHNICA Bucharest, Romania
Wall-OSS-0.5 Technical Report
May 29, 2026Ryan Yu, Pushi Zhang, Starrick Liu +24Zero-Shot Robotic Manipulation Cross-Embodiment Robot Learning
X Square Robot Team
From Pixels to Words -- Towards Native One-Vision Models at Scale
May 27, 2026Haiwen Diao, Jiahao Wang, Penghao Wu +18Vision-Language Models Video-Language Models
S-Lab, NTU · SenseTime Research · 3DLUT
GEM: Generative Supervision Helps Embodied Intelligence
May 27, 2026Ruowen Zhao, Bangguo Li, Zuyan Liu +9Vision-Language-Action Models Embodied AI
Tsinghua University · Tencent Hunyuan
Rethinking VLM Representation for VLA Initialization
May 25, 2026Weifeng Lin, Siyuan Huang, Hao Li +5VLM Adaptation Low-Rank Adaptation
1CUHK · 4ACE Robotics · 3Peking University +1
What Does the Caption Really Say? Counterfactual Phrase Intervention for Compositional Data Selection in Vision-Language Pretraining
May 21, 2026Hyejin Go, Semi Lee, Hyesong ChoiTraining Data Selection Vision-Language Pretraining
Soongsil University Seoul, Republic of Korea
Unified Pix Token And Word Token Generative Language Model
May 13, 2026Haun Leung, ZiNan WangImage Tokenization Self-Supervised Pre-Training
Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
May 13, 2026Zhaowei Wang, Lishu Luo, Haodong Duan +9Visual Question Answering Vision-Language Models
1CSE Department, HKUST · 2ByteDance Seed · *Work done at ByteDance Seed
CA-GCL: Cross-Anatomy Global-Local Contrastive Learning for Robust 3D Medical Image Understanding
May 13, 2026Hanwen Zhang, Yao Liu, Die Dai +4VLM Robustness Medical Image Anomaly Detection
University of Electronic Science and Technology of China, Chengdu, China · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
May 12, 2026DatologyAI, :, Siddharth Joshi +32Training Data Curation Large Vision-Language Models
DatologyAI Team1
Birds of a Feather Flock Together: Background-Invariant Representations via Linear Structure in VLMs
May 11, 2026Youssef Zaazou, Mark ThomasVLM Robustness Vision-Language Models
Independent Researcher
DINORANKCLIP: DINOv3 Distillation and Injection for Vision-Language Pretraining with High-Order Ranking Consistency
May 7, 2026Shuyang Jiang, Nan Yu, Yiming Zhang +2Contrastive Learning Vision-Language Alignment
University of California, Los Angeles · Aimaikj · HFIPS, Chinese Academy of Sciences +2
HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities
May 6, 2026Esra Dönmez, Pascal Tilli, Hsiu-Yu Yang +2Cross-Modal Learning Image Matching
Institute for Natural Language Processing, University of Stuttgart
Text-Conditional JEPA for Learning Semantically Rich Visual Representations
May 5, 2026Chen Huang, Xianhang Li, Vimal Thilak +2Joint-Embedding Predictive Architecture Predictive Representation Learning
Apple, Cupertino, United States
Scientific Domain Knowledge Improves Vision-Language Fundus Models
May 4, 2026Verena Jasmin Hallitschke, Carsten Eickhoff, Philipp BerensHealthcare Medical Diagnosis
1Hertie Institute for AI in Brain Health, University of Tübingen, Tübingen, Germany · Institute for Applied Medical Informatics, University of Tübingen, Tübingen, Germany · 2Tübingen AI Center, Tübingen, Germany
Let ViT Speak: Generative Language-Image Pre-training
May 1, 2026Yan Fang, Mengcheng Lan, Zilong Huang +7Cross-Modal Learning Vision Transformer
1Beijing Jiaotong University · 2ByteDance · 3Nanyang Technological University
Dynamic Cluster Data Sampling for Efficient and Long-Tail-Aware Vision-Language Pre-training
Apr 30, 2026Mingliang Liang, Zhuoran Liu, Arjen P. de Vries +1Long-Tail Learning Vision-Language Pretraining
Institute for Computing and Information Sciences, Radboud University, Nijmegen, the Netherlands · University of Amsterdam, Amsterdam, the Netherlands · Radboud University
Probing CLIP's Comprehension of 360-Degree Textual and Visual Semantics
Apr 27, 2026Hai Wang, Xiaochen Yang, Mingzhi Dong +1VLM Evaluation VLM Adaptation
Department of Statistical Science, University College London, UK · School of Mathematics and Statistics, University of Glasgow, UK · Department of Computer Science, University of Bath, UK.
PokeVLA: Empowering Pocket-Sized Vision-Language-Action Model with Comprehensive World Knowledge Guidance
Apr 22, 2026Yupeng Zheng, Xiang Li, Songen Gu +12Robot Foundation Models Efficient VLA Models
CASIA · TARS Robotics · Tsinghua University +3
Multilingual Training and Evaluation Resources for Vision-Language Models
Apr 20, 2026Daniela Baiamonte, Elena Fano, Matteo Gabburo +3VLM Evaluation Vision-Language Models
Aithlas
ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining
Mar 3, 2026Hanpeng Liu, Zidan Wang, Shuoxi Zhang +6Cross-Modal Learning Cross-Modal Alignment
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China · Li Auto Inc.
Xray-Visual Models: Scaling Vision models on Industry Scale Data
Feb 18, 2026Shlok Mishra, Tsung-Yu Lin, Linda Wang +24Vision Foundation Models Multimodal Pretraining
Meta-AI · MIT (work done while at Meta AI) · University of Maryland (work done while at Meta AI)
How to Train Your Long-Context Visual Document Model
Feb 16, 2026Austin VeselkaVisual Question Answering Vision-Language Models
LightOn
MTV: Revisiting Multi-Task Visual Representation Learning
Jan 20, 2026Shangzhe Di, Zhonghua Zhai, Weidi XieVisual Representation Learning Self-Supervised Visual Representation Learning
SAI, Shanghai Jiao Tong University · ByteDance Seed
CLIMP: Contrastive Language-Image Mamba Pretraining
Jan 11, 2026Nimrod Shabtay, Itamar Zimerman, Eli Schwartz +1Cross-Modal Learning VLM Robustness
Tel-Aviv University · IBM Research

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?

Nikos Giakoumoglou, Paschalis Giakoumoglou, Andreas Floros, Kleanthis Marios Papadopoulos+1

Imperial College London, London, UK · Centre for Research and Technology Hellas, Thessaloniki, Greece

Vision-Language Pretraining Contrastive Learning Synthetic Data Generation

Open paper AskPDF arXiv

Synthetic hard negatives generated in the representation space have proven effective for unimodal self-supervised learning, but transferring this idea to vision-language pretraining is not straightforward. We analyze six representation-space synthesis strategies and identify two failure modes in their transfer to vision-language pretraining: cross-modal constructions that produce overly easy negatives or pull them toward the query, and intra-modal constructions that incorporate the matched positive. We also observe logit-scale saturation when training with synthetic hard negatives and a learnable temperature, and find that fixing the temperature improves downstream performance. Using this geometric analysis we propose SNAP, which generates intra-modal hard negatives that never involve the positive from either modality, avoiding both failure modes entirely. SNAP is model-agnostic, requires no external generative models, and adds less than 10% training time overhead. Evaluated on top of CLIP and FLIP across multiple architectures and datasets, SNAP delivers consistent improvements on zero-shot retrieval, zero-shot classification, and linear probe evaluation.

Momentum

Related topics

Latest papers 50