Vision-Language Alignment

Follow this topic Ask about this topic

Vision-language alignment studies how models connect visual information with words, phrases, and longer descriptions, enabling tasks such as image-text retrieval, visual grounding, segmentation, and captioning. Common approaches include contrastive learning, prompt and adapter tuning, optimal transport, and fine-grained alignment between text tokens and image regions or video frames. Work also applies these methods to medical images, remote sensing, robotics, and multilingual settings, while developing benchmarks and diagnostics for compositional understanding and cross-modal correspondence. A central tension is that stronger semantic alignment can improve matching but may blur local details, amplify language bias, or weaken generalization across domains and tasks.

Latest papers

361 papers33 papers in the last four weeks, up 22% on the four weeks before. 0.2% of all new papers.

CycleCap: Improving VLMs Captioning Performance via Self-Supervised Cycle Consistency Fine-Tuning
Mar 18, 2026Marios Krestenitis, Christos Tzelepis, Konstantinos Ioannidis +5Cycle Consistency Image Captioning
Queen Mary, University of London · Centre for Research and Technology Hellas · City St George’s, University of London
Interpretable Cross-Domain Few-Shot Learning with Rectified Target-Domain Local Alignment
Mar 18, 2026Yaze Zhao, Yixiong Zou, Yuhua Li +1Cycle Consistency Source-Free Cross-Domain Few-Shot Learning
School of Computer Science and Technology, Huazhong University of Science and Technology
PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentation
Mar 18, 2026Jianjian Yin, Tao Chen, Yi Chen +4Open-Vocabulary Semantic Segmentation Vision-Language Alignment
Nanjing University of Science and Technology · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery · Nanjing Normal University +2
ACE-LoRA: Graph-Attentive Context Enhancement for Parameter-Efficient Adaptation of Medical Vision-Language Models
Mar 17, 2026M. Arda Aydın, Melih B. Yilmaz, Aykut Koç +1VLM Adaptation Medical VLMs
Bilkent University · National Magnetic Resonance Research Center (UMRAM)
Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification
Mar 14, 2026Jiachen Li, Xiaojin Gong, Dongping ZhangPerson Re-Identification Vision-Language Alignment
College of Information Science and Electronic Engineering, Zhejiang University · College of Information Engineering, China Jiliang University
Continual Learning with Vision-Language Models via Semantic-Geometry Preservation
Mar 12, 2026Chiyuan He, Zihuan Qiu, Fanman Meng +4Continual Learning Continual Learning for VLMs
School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
Noise-aware few-shot learning through bi-directional multi-view prompt alignment
Mar 12, 2026Lu Niu, Cheng XueFew-Shot Learning Visual Prompt Learning
Southeast University · AIIA, Ministry of Education, China
WikiCLIP: An Efficient Contrastive Baseline for Open-domain Visual Entity Recognition
Mar 10, 2026Shan Ning, Longtian Qiu, Jiaxuan Sun +1Vision-Language Alignment Multimodal Contrastive Learning
ShanghaiTech University, Shanghai, China · Lingang Laboratory, Shanghai, China · Shanghai Engineering Research Center of Intelligent Vision and Imaging
GeoAlignCLIP: Enhancing Fine-Grained Vision-Language Alignment in Remote Sensing via Multi-Granular Consistency Learning
Mar 10, 2026Xiao Yang, Ronghao Fu, Zhuoran Duan +3Vision-Language Alignment Remote Sensing Image Understanding
College of Computer Science and Technology, Jilin University, Changchun 130012, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University
BiCLIP: Domain Canonicalization via Structured Geometric Transformation
Mar 9, 2026Pranav Mantini, Shishir K. ShahVLM Adaptation Domain Adaptation
University of Houston, Department of Computer Science · The University of Oklahoma, School of Computer Science
Local-Global Prompt Learning via Sparse Optimal Transport
Mar 9, 2026Deniz Kizaroğlu, Ülku Tuncer Küçüktas, Emre Çakmakyurdu +1VLM Adaptation Visual Prompt Learning
Graduate School of Informatics, Middle East Technical University, Ankara, Turkey · Department of Electrical and Electronics Engineering, Gazi University, Ankara, Turkey
VIVECaption: A Split Approach to Caption Quality Improvement
Mar 8, 2026Varun Ananth, Baqiao Liu, Haoran CaiImage Captioning Evaluation Image Captioning
Adobe Inc.
VisualScratchpad: Inference-time Visual Concepts Analysis in Vision Language Models
Mar 7, 2026Hyesu Lim, Jinho Choi, Taekyung Kim +3VLM Interpretability Vision-Language Alignment
KAIST AI · NAVER AI Lab
Mind the Discriminability Trap in Source-Free Cross-domain Few-shot Learning
Mar 7, 2026Zhenyu Zhang, Yixiong Zou, Yuhua Li +2Source-Free Cross-Domain Few-Shot Learning Few-Shot Learning
School of Computer Science and Technology, Huazhong University of Science and Technology · National Key Laboratory for Multimedia Information Processing, Peking University
Information-Theoretic Constraints for Continual Vision-Language-Action Alignment
Mar 6, 2026Libang Zhao, Qixin Zeng, Hongyin Zhang +1Continual Learning for VLA Models Vision-Language Alignment
Westlake University, China · University of Southampton, UK
Learning Adaptive and Visually Aligned Conditions for Generative Zero-Shot Learning
Mar 6, 2026Haojie Pu, Zhuoming Li, Yongbiao Gao +3Zero-Shot Learning Vision-Language Alignment
School of Computer Science and Engineering, Southeast University, Nanjing, China
QATMA: Quantization-Aware Training with Multimodal Alignment for Open-Vocabulary Object Detection
Mar 6, 2026Jinyeong Park, Donghwa Kang, Seunghwan An +4VLM Quantization Quantization-Aware Training
Incheon National University, Incheon, South Korea · KAIST, Daejeon, South Korea · University of Seoul, Seoul, South Korea
A novel Framework for Open-Vocabulary Multi-Object Recognition using CLIP
Mar 6, 2026Wei Yu Chen, Ying DaiOpen-Vocabulary Object Detection Vision-Language Alignment
Faculty of Software and Information Science Iwate Prefectural University Takizawa, Japan
HERO: Hierarchical Embedding-Refinement for Open-Vocabulary Temporal Sentence Grounding in Videos
Mar 6, 2026Tingting Han, Xinsong Tao, Yufei Yin +3Temporal Video Grounding Vision-Language Alignment
Laboratory of Complex Systems Modeling and Simulation, School of Computer Science and Technology, Hangzhou Dianzi University · Zhejiang Key Laboratory of Space Information Sensing and Transmission, Hangzhou Dianzi University · Department of Psychological and Cognitive Sciences, Tsinghua University
Reclaiming Lost Text Layers for Source-Free Cross-Domain Few-Shot Learning
Mar 5, 2026Zhenyu Zhang, Guangyao Chen, Yixiong Zou +2Source-Free Cross-Domain Few-Shot Learning Few-Shot Learning
School of Computer Science and Technology, Huazhong University of Science and Technology · National Key Laboratory for Multimedia Information Processing, Peking University
Aligning the True Semantics: Constrained Decoupling and Distribution Sampling for Cross-Modal Alignment
Mar 5, 2026Xiang Ma, Lexin Fang, Litian Xu +1Cross-Modal Alignment Vision-Language Alignment
Shandong University, Jinan, China · The University of Exeter, Exeter, United Kingdom
VisionPangu: A Compact and Fine-Grained Multimodal Assistant with 1.7B Parameters
Mar 5, 2026Jiaxin Fan, Wenpo SongVision-Language Models Multimodal Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University
PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest
Mar 3, 2026Josh Beal, Eric Kim, Jinfeng Rao +3Multimodal Embedding Multimodal IR
Pinterest Inc., San Francisco, CA, USA
HDINO: A Concise and Efficient Open-Vocabulary Detector
Mar 3, 2026Hao Zhang, Yiqun Wang, Qinran Lin +2Open-Vocabulary Object Detection Multimodal Object Detection
Chongqing University
CAPT: Confusion-Aware Prompt Tuning for Reducing Vision-Language Misalignment
Mar 3, 2026Maoyuan Shao, Yutong Gao, Xinyang Huang +3VLM Adaptation Vision-Language Alignment
School of Information Engineering, Minzu University of China · School of Artificial Intelligence, Beijing University of Posts and Telecommunications · National Library of China, Beijing, China +1
Unified Vision-Language Modeling via Concept Space Alignment
Mar 1, 2026Yifu Qiu, Paul-Ambroise Duquenne, Holger SchwenkVision-Language Models Vision-Language Alignment
University of Edinburgh · FAIR at Meta
Vision-Language Feature Alignment for Road Anomaly Segmentation
Mar 1, 2026Zhuolin He, Jiacheng Tang, Jian Pu +1Image Segmentation Autonomous Driving Perception
School of Computer Science, Fudan University, Shanghai 200433, China · Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, 200433, China
TokenCom: Vision-Language Model for Multimodal and Multitask Token Communications
Feb 28, 2026Feibo Jiang, Siwei Tu, Li Dong +5Joint Source-Channel Coding Vision-Language Models
Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing, Hunan Normal University, Changsha, China. · School of Information Science and Engineering, Hunan Normal University, Changsha, China. · Changsha Social Laboratory of Artificial Intelligence, Hunan University of Technology and Business, Changsha, China. +5
PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models
Feb 28, 2026Yuanhao Su, Shaofeng Zhang, Xiaosong Jia +13D VLMs Vision-Language Alignment
University of Science and Technology of China · Fuzhou University · Fudan University +1
Half-Truths Break Similarity-Based Retrieval
Feb 27, 2026Bora Kargi, Arnas Uselis, Seong Joon OhContrastive Learning for VLMs Vision-Language Models
University of Tübingen, Tübingen AI Center · Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA)

Mar 18, 2026J/K move · Enter open · S save

CycleCap: Improving VLMs Captioning Performance via Self-Supervised Cycle Consistency Fine-Tuning

Marios Krestenitis, Christos Tzelepis, Konstantinos Ioannidis, Stefanos Vrochidis+4

Queen Mary, University of London · Centre for Research and Technology Hellas · City St George’s, University of London

Cycle Consistency Image Captioning Vision-Language Models Vision-Language Alignment VLM Hallucination Mitigation

Open paper AskPDF arXiv

Visual-Language Models (VLMs) have achieved remarkable progress in image captioning, visual question answering, and visual reasoning. Yet they remain prone to vision-language misalignment, often producing overly generic or hallucinated descriptions. Existing approaches address this via instruction tuning-requiring costly, large-scale annotated datasets or via complex test-time frameworks for caption refinement. In this work, we revisit image-text alignment through the lens of cycle consistency: given an image and a caption generated by an image-to-text model, the backward mapping through a text-to-image model should reconstruct an image that closely matches the original. In our setup, a VLM serves as the image-to-text component, while a pre-trained text-to-image model closes the loop by reconstructing the image from the generated caption. Building on this, we introduce CycleCap, a fine-tuning scheme to improve image captioning using Group Relative Policy Optimization (GRPO) with a reward based on the similarity between the original and reconstructed images, computed on-the-fly. Unlike previous work that uses cycle consistency loss for preference dataset construction, our method leverages cycle consistency directly as a self-supervised training signal. This enables the use of raw images alone, eliminating the need for curated image-text datasets, while steering the VLM to produce more accurate and grounded text descriptions. Applied to four VLMs ranging from 1B to 7B parameters, CycleCap yields consistent improvements across captioning and hallucination benchmarks, surpassing state-of-the-art methods that rely on supervised cycle consistency training.