Vision-Language Alignment

Ask about this topic

Overview

Vision-language alignment studies how models connect visual information with words, phrases, and longer descriptions, enabling tasks such as image-text retrieval, visual grounding, segmentation, and captioning. Common approaches include contrastive learning, prompt and adapter tuning, optimal transport, and fine-grained alignment between text tokens and image regions or video frames. Work also applies these methods to medical images, remote sensing, robotics, and multilingual settings, while developing benchmarks and diagnostics for compositional understanding and cross-modal correspondence. A central tension is that stronger semantic alignment can improve matching but may blur local details, amplify language bias, or weaken generalization across domains and tasks.

Latest papers

CardsList

361 papers33 papers in the last four weeks, up 22% on the four weeks before. 0.2% of all new papers.

  1. T-REN: Learning Text-Aligned Region Tokens Improves Dense Vision-Language Alignment and Scalability

    Apr 20, 2026Savya Khosla, Sethuraman T, Aryan Chadha +2VLM AdaptationOpen-Vocabulary Semantic Segmentation

    University of Illinois Urbana-Champaign

  2. S2H-DPO: Hardness-Aware Preference Optimization for Vision-Language Models

    Apr 20, 2026Nitish Shukla, Surgan Jandial, Arun RossDirect Preference OptimizationVisual Reasoning

    Michigan State University · Carnegie Mellon University

  3. Towards Robust Text-to-Image Person Retrieval: Multi-View Reformulation for Semantic Compensation

    Apr 20, 2026Chao Yuan, Yujian Zhao, Haoxuan Xu +1VLM RobustnessVision-Language Retrieval

    Beihang University

  4. When Vision-Language Models Judge Without Seeing: Exposing Informativeness Bias

    Apr 20, 2026Xiaohan Zou, Roshan Sridhar, Mohammadtaher Safarzadeh +1VLM EvaluationVision-Language Alignment

    The Pennsylvania State University · Oracle AI

  5. Dynamic Visual-semantic Alignment for Zero-shot Learning with Ambiguous Labels

    Apr 20, 2026Jiangnan Li, Linqing Huang, Xiaowen Yan +3Zero-Shot LearningNoisy-Label Learning

    College of Computer Science and Technology, Qingdao University · School of Computer Science, Shanghai JiaoTong University · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)

  6. Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability

    Apr 19, 2026Lijie ZhouCross-Modal LearningVLM Robustness

    School of Computer Science University of Nottingham Ningbo China Ningbo, China

  7. Motion-Guided Semantic Alignment with Negative Prompts for Zero-Shot Video Action Recognition

    Apr 18, 2026Yiming Wang, Frederick W. B. Li, Jingyun WangRepresentation LearningVideo Action Recognition

    Department of Computer Science, Durham University

  8. MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation

    Apr 18, 2026Bo Li, Ningyuan Deng, Tianyu Dong +3Image Text TranslationFine-Tuning

    School of Software, Tsinghua University, Beijing, China · Baidu Inc., Beijing, China · School of Information Resource Management, Renmin University of China,Beijing, China +2

  9. Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow

    Apr 17, 2026Chengxin Liu, Wonseok Choi, Chenshuang Zhang +1Vision-Language AlignmentVLM Inference

    KAIST · POSTECH

  10. PLAF: Pixel-wise Language-Aligned Feature Extraction for Efficient 3D Scene Understanding

    Apr 17, 2026Junjie Wen, Junlin He, Fei Ma +13D Scene Representation3D Representation Learning

    Pengcheng Laboratory, Shenzhen, 518108, China · The School of Intelligent Engineering, Sun Yat-Sen University-ShenzhenCampus, Shenzhen, Guangdong, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.

  11. Concept-wise Attention for Fine-grained Concept Bottleneck Models

    Apr 17, 2026Minghong Zhong, Guoshuai Zou, Kanghao Chen +2Concept Bottleneck ModelsVision-Language Alignment

    Sun Yat-sen University · Peng Cheng Laboratory · Key Laboratory of Machine Intelligence and Advanced Computing, MOE +1

  12. Switch-KD: Visual-Switch Knowledge Distillation for Vision-Language Models

    Apr 16, 2026Haoyi Sun, Xiaoxiao Wang, Ning Mao +5Cross-Modal Knowledge DistillationVision-Language Models

    Li Auto Inc.

  13. MApLe: Multi-instance Alignment of Diagnostic Reports and Large Medical Images

    Apr 15, 2026Felicia Bader, Philipp Seeböck, Anastasia Bartashova +2Radiology VLMsVision-Language Alignment

    Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Austria · Comprehensive Center for Artificial Intelligence in Medicine, Medical University of Vienna, Austria · Medical Anomaly Detection (MANO) Group, Computational Imaging Research (CIR), Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Austria +1

  14. Concrete Jungle: Towards Concreteness Paved Contrastive Negative Mining for Compositional Understanding

    Apr 14, 2026Eun Woo Im, Dhruv Madhwal, Vivek GuptaCompositional Reasoning in VLMsVLM Reasoning

    Arizona State University

  15. Reading Between the Pixels: Linking Text-Image Embedding Alignment to Typographic Attack Success on Vision-Language Models

    Apr 14, 2026Ravikumar Balakrishnan, Sanket Mendapara, Ankit GargAdversarial Attacks on VLMsVLM Robustness

    Cisco Systems

  16. TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment

    Apr 13, 2026Bingyi Cao, Koert Chen, Kevis-Kokitsi Maninis +16Vision-Language PretrainingVision-Language Alignment

    Google DeepMind · xAI · Epsilon Health +1

  17. Revisiting Compositionality in Dual-Encoder Vision-Language Models: The Role of Inference

    Apr 13, 2026Imanol Miranda, Ander Salaberria, Eneko Agirre +1Compositional Reasoning in VLMsVision-Language Models

    HiTZ Center – Ixa, University of the Basque Country (UPV/EHU)

  18. Anthropogenic Regional Adaptation in Multimodal Vision-Language Model

    Apr 13, 2026Samuel Cahyawijaya, Peerat Limkonchotiwat, Tack Hwa Wong +45VLM AdaptationVision-Language Alignment

  19. MedP-CLIP: Medical CLIP with Region-Aware Prompt Integration

    Apr 13, 2026Jiahui Peng, He Yao, Jingwen Li +9Promptable Image SegmentationMultimodal Pretraining

    aSichuan University, Chengdu, 610065, China · bXinjiang University, Urumqi, 830046, China · cAlibaba Group, Hangzhou, 310023, China +3

  20. Turning Generators into Retrievers: Unlocking MLLMs for Natural Language-Guided Geo-Localization

    Apr 12, 2026Yuqi Chen, Xiaohan Zhang, Ahmad Arrabi +3Cross-View Geo-LocalizationVision-Language Retrieval

    Vermont Artificial Intelligence Lab, Department of Computer Science, University of Vermont · Intelligent Machines Lab, Department of Artificial Intelligence, Information Technology University · Institute of Artificial Intelligence, University of Central Florida

  21. See Fair, Speak Truth: Equitable Attention Improves Grounding and Reduces Hallucination in Vision-Language Alignment

    Apr 10, 2026Mohammad Anas Azeez, Ankan Deria, Zohaib Hasan Siddiqui +5VLM Hallucination MitigationObject Hallucination in VLMs

    Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia · Macquarie University, Sydney, Australia +2

  22. UniSemAlign: Text-Prototype Alignment with a Foundation Encoder for Semi-Supervised Histopathology Segmentation

    Apr 10, 2026Le-Van Thai, Tien Dat Nguyen, Hoai Nhan Pham +3Histopathology Image SegmentationSemi-Supervised Medical Image Segmentation

    AI VIETNAM Lab, Vietnam · Hanoi University of Science and Technology, Vietnam

  23. InstAP: Instance-Aware Vision-Language Pre-Train for Spatial-Temporal Understanding

    Apr 9, 2026Ashutosh Kumar, Rajat Saini, Jingjing Pan +5Vision-Language PretrainingVision-Language Alignment

    Woven by Toyota

  24. DSCA: Dynamic Subspace Concept Alignment for Lifelong VLM Editing

    Apr 9, 2026Gyanendra Das, Sai Satyam JenaContinual Learning for VLMsVision-Language Alignment

    Zynix AI, FL, USA

  25. MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models

    Apr 7, 2026Han Jang, Junhyeok Lee, Heeseong Eum +1HealthcareMedical VLMs

    Seoul National University · Department of Radiology, Seoul National University Hospital · The Advanced Imaging and Computational Neuroimaging (AICON) Laboratory +2

  26. Grounding Hierarchical Vision-Language-Action Models Through Explicit Language-Action Alignment

    Apr 7, 2026Theodor Wulff, Federico Tavella, Rahul Singh Maharjan +2Vision-Language-Action ModelsVision-Language Alignment

    The University of Manchester, Manchester, United Kingdom

  27. Learning What Matters: Dynamic Dimension Selection and Aggregation for Interpretable Vision-Language Reward Modeling

    Apr 7, 2026Qiyuan Chen, Hongsen Huang, Jiahe Chen +6Multimodal Reward ModelingVision-Language Alignment

    Zhejiang University · Soochow Securities Co.,Ltd. · HKUST(GZ) +1

  28. Spatially-Weighted CLIP for Street-View Geo-localization

    Apr 6, 2026Ting Han, Fengjiao Li, Chunsong Chen +3Geospatial Representation LearningVisual Place Recognition

    School of Geospatial Engineering and Science, Sun Yat-Sen University, Zhuhai, China · School of System Science and Engineering, Sun Yat-Sen University, Guangzhou, China

  29. Steerable Visual Representations

    Apr 2, 2026Jona Ruthardt, Manu Gaur, Deva Ramanan +2Visual Representation LearningVision-Language Alignment

    University of Technology Nuremberg · Carnegie Mellon University · International Institute of Information Technology, Hyderabad

  30. ProVG: Progressive Visual Grounding via Language Decoupling for Remote Sensing Imagery

    Apr 2, 2026Ke Li, Ting Wang, Di Wang +4Remote Sensing Visual GroundingRemote Sensing VLMs

    Xidian University, China · UC San Diego, USA · SUST, China