Vision-Language Alignment

Ask about this topic

Overview

Vision-language alignment studies how models connect visual information with words, phrases, and longer descriptions, enabling tasks such as image-text retrieval, visual grounding, segmentation, and captioning. Common approaches include contrastive learning, prompt and adapter tuning, optimal transport, and fine-grained alignment between text tokens and image regions or video frames. Work also applies these methods to medical images, remote sensing, robotics, and multilingual settings, while developing benchmarks and diagnostics for compositional understanding and cross-modal correspondence. A central tension is that stronger semantic alignment can improve matching but may blur local details, amplify language bias, or weaken generalization across domains and tasks.

Latest papers

CardsList

361 papers33 papers in the last four weeks, up 22% on the four weeks before. 0.2% of all new papers.

  1. Benchmarking and Mechanistic Analysis of Vision-Language Models for Cross-Depiction Assembly Instruction Alignment

    Apr 1, 2026Zhuchenyang Liu, Yao Zhang, Yu XiaoVision-Language AlignmentVLM Robustness

    Aalto University, Espoo, Finland

  2. Hierarchical Pre-Training of Vision Encoders with Large Language Model

    Mar 31, 2026Eugene Lee, Ting-Yu Chang, Jui-Huang Tsai +2Vision-Language PretrainingLarge Vision-Language Models

    University of Cincinnati · National Yang Ming Chiao Tung University

  3. M-MiniGPT4: Multilingual VLLM Alignment via Translated Data

    Mar 31, 2026Seung Hun Han, Youssef Mohamed, Mohamed ElhoseinyVision-Language ModelsVision-Language Alignment

    MBZUAI · KAUST

  4. AGFT: Alignment-Guided Fine-Tuning for Zero-Shot Adversarial Robustness of Vision-Language Models

    Mar 31, 2026Yubo Cui, Xianchao Guan, Zijun Xiong +1VLM RobustnessVision-Language Alignment

    Harbin Institute of Technology, Shenzhen, China · Shenzhen Loop Area Institute, Shenzhen, China

  5. LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generation

    Mar 29, 2026Shentong Mo, Sukmin YunVision-Language AlignmentUnified Multimodal Models

    Department of Machine Learning, CMU, USA · Department of Machine Learning, MBZUAI, UAE · Department of Artificial Intelligence, Hanyang University ERICA, South Korea

  6. Difference Feedback: Generating Multimodal Process-Level Supervision for VLM Reinforcement Learning

    Mar 29, 2026Feiding, Yongkang Zhang, Yuhao Liao +10RL for VLMsReasoning for VLMs

    Shenzhen International Graduate School, Tsinghua University, China

  7. On the Spectral Geometry of Cross-Modal Representations: A Functional Map Diagnostic for Multimodal Alignment

    Mar 28, 2026Krisanu SarkarFunctional MapsCross-Modal Alignment

    Indian Institute of Technology Bombay, Mumbai, India

  8. SDDF: Specificity-Driven Dynamic Focusing for Open-Vocabulary Camouflaged Object Detection

    Mar 27, 2026Jiaming Liang, Yifeng Zhan, Chunlin Liu +6Camouflaged Object DetectionOpen-Vocabulary Object Detection

    Shenzhen University · The Hong Kong University of Science and Technology (Guangzhou)

  9. No Hard Negatives Required: Concept Centric Learning Leads to Compositionality without Degrading Zero-shot Capabilities of Contrastive Models

    Mar 26, 2026Hai X. Pham, David T. Hoffmann, Ricardo Guerrero +1Vision-Language ModelsCompositional Reasoning in VLMs

    Samsung AI Center Cambridge, UK

  10. EagleNet: Energy-Aware Fine-Grained Relationship Learning Network for Text-Video Retrieval

    Mar 26, 2026Yuhan Chen, Pengwen Dai, Chuan Wang +2Vision-Language RetrievalVision-Language Alignment

    Sun Yat-sen University, Guangzhou, China · Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · Shenzhen Key Laboratory of Adversarial Artificial Intelligence, Shenzhen, China +2

  11. Learning to Rank Caption Chains for Video-Text Alignment

    Mar 26, 2026Ansel Blume, Burak Uzkent, Shalini Chaudhuri +1Vision-Language AlignmentPreference Optimization

    University of Illinois Urbana-Champaign · Amazon Prime Video

  12. CVA: Context-aware Video-text Alignment for Video Temporal Grounding

    Mar 26, 2026Sungho Moon, Seunghun Lee, Jiwan Seo +1Temporal Video GroundingVision-Language Alignment

    Daegu Gyeongbuk Institute of Science and Technology (DGIST), Republic of Korea

  13. Explaining, Verifying, and Aligning Semantic Hierarchies in Vision-Language Model Embeddings

    Mar 26, 2026Gesina Schwalbe, Mert Keser, Moritz Bayerkuhnlein +8VLM InterpretabilityVision-Language Alignment

    Ulm University, Germany · University of Lübeck, Germany · Technical University of Munich, Germany +4

  14. Evaluating Vision-Language Models for Image Quality Assessment using Psychophysical Data

    Mar 25, 2026Imran Mehmood, Imad Ali Shah, Ming Ronnier Luo +1Image Quality AssessmentVLM Evaluation

    School of Engineering, University of Galway, Ireland · Lero the Research Ireland Centre for Software, Ireland · State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang University, Hangzhou, China.

  15. CliPPER: Contextual Video-Language Pretraining on Long-form Intraoperative Surgical Procedures for Event Recognition

    Mar 25, 2026Florian Stilz, Vinkle Srivastav, Nassir Navab +1Surgical Video UnderstandingVideo-Language Models

    University of Strasbourg, CNRS, INSERM, ICube, UMR7357, France · IHU Strasbourg, France · Technical University of Munich, Germany

  16. Project and Mix: Task-Semantic Prototypes for Few-Shot Image Classification

    Mar 25, 2026Dipam Goswami, Simone Magistri, Gido M. van de Ven +4Prototype-Based LearningVLM Adaptation

    Department of Computer Science, Universitat Autònoma de Barcelona, Spain · Computer Vision Center, Barcelona, Spain · Media Integration and Communication Center, University of Florence, Italy +3

  17. Unleashing Vision-Language Semantics for Deepfake Video Detection

    Mar 25, 2026Jiawen Zhu, Yunqi Miao, Xueyi Zhang +2Deepfake DetectionVision-Language Alignment

    Singapore Management University, Singapore · The University of Warwick, UK · Nanyang Technological University, Singapore +1

  18. Decompose and Transfer: CoT-Prompting Enhanced Alignment for Open-Vocabulary Temporal Action Detection

    Mar 25, 2026Sa Zhu, Wanqian Zhang, Lin Wang +4Temporal Action DetectionTemporal Video Understanding

    Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · State Key Laboratory of Cyberspace Security Defense +2

  19. SilLang: Improving Gait Recognition with Silhouette Language Encoding

    Mar 25, 2026Ruiyi Zhan, Guozhen Peng, Canyu Chen +2Gait RecognitionVisual Representation Learning

    Beihang University · Tsinghua University

  20. Looking Beyond the Window: Global-Local Aligned CLIP for Training-free Open-Vocabulary Semantic Segmentation

    Mar 24, 2026ByeongCheol Lee, Hyun Seok Seong, Sangeek Hyun +3Open-Vocabulary Semantic SegmentationSliding-Window Attention

    Sungkyunkwan University

  21. Cluster-Wise Spatio-Temporal Masking for Efficient Video-Language Pretraining

    Mar 24, 2026Weijun Zhuang, Yuqing Huang, Weikang Meng +5Video-Language ModelsVision-Language Alignment

    Harbin Institute of Technology · Pengcheng Laboratory

  22. WiFi2Cap: Semantic Action Captioning from Wi-Fi CSI via Limb-Level Semantic Alignment

    Mar 24, 2026Tzu-Ti Wei, Chu-Yu Huang, Yu-Chee Tseng +1Wi-Fi SensingHuman Activity Recognition

    PAIRLab, College of AI, National Yang Ming Chiao Tung University, Taiwan, R.O.C

  23. ACPO: Counteracting Likelihood Displacement in Vision-Language Alignment with Asymmetric Constraints

    Mar 23, 2026Kaili Huang, Hongming Zhang, Rui Shen +4Vision-Language AlignmentPreference Optimization

    SenseTime Research

  24. Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models

    Mar 23, 2026Hayeon Kim, Ji Ha Jang, Junghun James Kim +1Vision-Language ModelsHierarchical Representation Learning

    Dept. of Electrical and Computer Engineering, Seoul National University, Republic of Korea · INMC & IPAI, Seoul National University, Republic of Korea

  25. PEARL: Geometry Aligns Semantics for Training-Free Open-Vocabulary Semantic Segmentation

    Mar 23, 2026Gensheng Pei, Xiruo Jiang, Xinhao Cai +3Graph-Based Label PropagationOpen-Vocabulary Semantic Segmentation

    Department of Electrical and Computer Engineering, Sungkyunkwan University · School of Computing and Artificial Intelligence, Southwest Jiaotong University · School of Computer Science and Engineering, Nanjing University of Science and Technology

  26. ALADIN:Attribute-Language Distillation Network for Person Re-Identification

    Mar 23, 2026Wang Zhou, Boran Duan, Haojun Ai +2Person Re-IdentificationCross-Modal Knowledge Distillation

    Wuhan University No.299 Bayi Road, 430072, PR China

  27. Mitigating Objectness Bias and Region-to-Text Misalignment for Open-Vocabulary Panoptic Segmentation

    Mar 22, 2026Nikolay Kormushev, Josip Šarić, Matej KristanPanoptic SegmentationVision-Language Alignment

    University of Ljubljana Faculty of Comp. and Inf. Science · ETH Zurich Dept. of Computer Science · University of Zagreb Faculty of Elec. Eng. and Computing

  28. SkinCLIP-VL: Consistency-Aware Vision-Language Learning for Multimodal Skin Cancer Diagnosis

    Mar 22, 2026Zhixiang Lu, Shijie Xu, Kaicheng Yan +6Medical VLMsVLM Adaptation

    Xi’an Jiaotong-Liverpool University, China · University of Liverpool, United Kingdom

  29. Cross-modal Fuzzy Alignment Network for Text-Aerial Person Retrieval and A Large-scale Benchmark

    Mar 21, 2026Yifei Deng, Chenglong Li, Yuyang Zhang +2Image-Text RetrievalCross-Modal Alignment

    State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology · School of Computer Science and Technology, Anhui University · The University of Hong Kong +1

  30. HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction

    Mar 20, 2026Ruicheng Yuan, Zhenxuan Zhang, Anbang Wang +5Computational PathologyMedical Report Generation

    College of Computer Science and Electronic Engineering, Hunan University, Changsha, Hunan, China · Department of Bioengineering and Imperial-X, Imperial College London, London, UK · Department of Pathology, Xiangtan Maternal and Child Health Hospital, Xiangtan, Hunan, China +1