Vision-Language Pretraining

Overview

Vision-language pretraining teaches models to connect visual content with language, supporting image-text retrieval, zero-shot classification, question answering, and applications such as medical image analysis. Common approaches include contrastive image-text alignment, caption generation, masked or predictive feature learning, and region-level grounding, trained on web captions, curated datasets, or domain-specific reports. Researchers evaluate transfer on classification and retrieval benchmarks such as ImageNet and COCO, as well as on dense perception, visual question answering, documents, and clinical tasks. A central tension is that global alignment scales well but can miss fine details and composition, while local or generative objectives offer richer supervision but vary in benefit across tasks and settings.

Latest papers

CardsList

82 papers3 papers in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

  1. Vision-Grounded Semantic Learning and Alignment for Scalable Multilingual Text-Based Person Retrieval

    Nov 11, 2026Xinyu Zhou, Fan Li, Shasha Chen +4Text-Based Person SearchVision-Language Alignment

  2. Findings-aware Language-Image Pretraining for CT Volume Interpretation

    Nov 2, 2026Takuro Shimaya, Junya Inoh, Masahiro SaikoMedical VLMsMedical Image Representation Learning

  3. ALPS: Visual State Encoders for World Models

    Oct 21, 2026Yanqing Liu, Letian Zhang, Chengzhi Liu +5World Model LearningVision-Language Pretraining

  4. Beyond Sparse Captions: Aligning Slide-Level Text and Patch-Level Vision in Pathology

    Oct 12, 2026Daniel Shao, Dongmin Bang, Luca L. Weishaupt +6Computational PathologyMedical VLMs

  5. What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?

    Oct 7, 2026Nikos Giakoumoglou, Paschalis Giakoumoglou, Andreas Floros +2Vision-Language PretrainingContrastive Learning

  6. VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training

    Sep 29, 2026Yuanwei Hu, Bo Peng, Yuheng Jia +3ClusteringDeep Clustering

  7. SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis

    Sep 28, 2026Hangyul Yoon, Hyungyung Lee, Edward Choi +1Contrastive LearningRadiology VLMs

  8. Clinical Trajectory Alignment for Medical Vision-Language Pre-training

    Sep 28, 2026Huimin Yan, Xian Yang, Zhi Wang +1Multimodal PretrainingLongitudinal Medical Image Analysis

  9. Region-Level Black-Box Defense Against Stealthy Embedding-Space Backdoors in CLIP

    Sep 25, 2026Ahmed Abdelnaby, Mohamed ElmahallawyBackdoor DefenseVLM Robustness

  10. TDDN: Text-aligned Diffused DINO Network for Puzzle Understanding

    Sep 7, 2026Harsha Patnala, Debopriyo Banerjee, Ayush Sunil Munot +1Visual ReasoningVision-Language Alignment

  11. Bigger Text Encoders Can Hurt CLIP Zero-Shot Performance

    Sep 4, 2026Samir Char, Carles Domingo-Enrich, Randall BalestrieroVision-Language ModelsVision-Language Pretraining

  12. On the Design Fundamentals of Pixel Text Representation Learning

    Sep 1, 2026Chaohao Yuan, Ruifeng Yuan, Zhuoxu Huang +4Visual Representation LearningVision-Language Alignment

  13. An Integrated Vision-and-Language Pretraining (VLP) and Visual Question Answering (VQA) model to Automate Nondestructive Evaluation Image Analysis

    Aug 29, 2026Mehrdad Shafiei Dizaji, Hoda AzariNon-Destructive TestingImage Captioning

  14. VIVAS: Vitalizing Visual Perception in VLM Pre-training via Vision-language Unified Autoregressive Supervision

    Aug 25, 2026Zhehan Kan, Yubo Zhu, Xinghua Jiang +10Vision-Language PretrainingFine-Grained Visual Perception

  15. What Does CLIP Learn for Regional Geolocalization? Probing Visual Cues and Scene Configuration After Adaptation

    Aug 22, 2026Changyu Lee, Yeonsoo Park, Abdullah Alfarrarjeh +1Contrastive Language-Image PretrainingGeospatial Representation Learning

  16. V-REX: Efficient Specialist VLM Training for Veterinary X-Rays

    Aug 20, 2026Tim Elsner, Nicole McNally, Andre Dourson +1Efficient VLM TrainingVision-Language Models

  17. CL4D: Contrastive Language-4D Pretraining for Vision-Language Reasoning in Dynamic Scenes

    Aug 19, 2026Kumal Hewagamage, Isuranga Senavirathne, Sasika Amarasinghe +4Vision-Language ModelsVision-Language Alignment

  18. A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports

    Jul 30, 2026Jia Yu, Yan Zhu, Yili He +12EndoscopyMedical VLMs

  19. DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes

    Jul 27, 2026Jiahao Xie, Zhongbin Guo, Qianle Wang +4Training Data CurationData Mixture Optimization

  20. UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models

    Jul 25, 2026Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati +4Vision-Language ModelsEfficient VLM Inference

  21. AspectCLIP: Optimizing CLIP Representation Space via Aspect-Guided Consistency Regularization

    Jul 15, 2026Yiyang Yao, Shanglin Liu, Jianming Lv +4Vision-Language AlignmentMultimodal Contrastive Learning

  22. MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

    Jul 13, 2026Yuliang Liu, Zhang Li, Ziyang Zhang +11Document ParsingMultimodal Foundation Models

  23. SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception

    Jul 13, 2026Mingjie Xie, Guangjun He, Dongli Xu +5VLM RobustnessVision-Language Grounding

  24. LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives

    Jul 1, 2026Lukas Kuhn, Giuseppe Serra, Randall Balestriero +1Visual Representation LearningVision-Language Pretraining

  25. SonoCLIP: Mask-Guided Region-Aware Vision-Language Pretraining for Fetal Ultrasound Analysis

    Jun 28, 2026Hang Su, Chao Sun, Zhaofan Li +3Medical Imaging Foundation ModelsUltrasound Imaging

  26. DataComp-VLM: Improved Open Datasets for Vision-Language Models

    Jun 26, 2026Matteo Farina, Vishaal Udandarao, Thao Nguyen +33VLM EvaluationTraining Data Curation