VLM Adaptation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

34 papers in the last four weeks, up 209% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 395

All topics

Variational Adapter for Cross-modal Similarity Representation
May 29, 2026WenZhang Wei, Zhipeng Gui, Dehua Peng +2Cross-Modal Representation Learning VLM Adaptation
School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China · School of Data Science and Engineering, East China Normal University, Shanghai 200062, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
LoMo: Local Modality Substitution for Deeper Vision-Language Fusion
May 28, 2026Feng Han, Zhixiong Zhang, Zheming Liang +2Cross-Modal Learning VLM Robustness
Fudan University · Shanghai Innovation Institute · Shanghai Jiao Tong University +1
Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning
May 28, 2026Shuai Yi, Yixiong Zou, Yuhua Li +1Source-Free Domain Adaptation Few-Shot Domain Adaptation
Institute of Artificial Intelligence, Huazhong University of Science and Technology, Wuhan, China · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China.
GAP3D: Generative Alignment of VLM Latents to Patch-Level Embeddings for 3D Generation
May 27, 2026Polytimi Anna Gkotsi, Andrii Zadaianchuk, Mohammad Mahdi DerakhshaniDiffusion Model Alignment 3D Asset Generation
University of Amsterdam, The Netherlands
AREA: Attribute Extraction and Aggregation for CLIP-Based Class-Incremental Learning
May 27, 2026Zhen-Hao Xie, Yu-Cheng Shi, Da-Wei ZhouClass-Incremental Learning Vision-Language Models
School of Artificial Intelligence, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
EntroAD: Structural Entropy-Guided Prompt Adaptation for Zero-Shot Anomaly Detection
May 27, 2026Xinyu Zhao, Qingyun Sun, Jiayi Luo +1VLM Adaptation Medical Image Anomaly Detection
Beihang University Beijing, China
Toward Semantic-Agnostic and Shape-Aware Vision-Language Segmentation Models
May 27, 2026Corentin Seutin, Mohamed Amine Ettaki, Michaël Clément +2VLM Adaptation Text-Guided Image Segmentation
Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, France · Univ. Bordeaux, CNRS, Bordeaux INP, IMS, UMR 5218, France
FedMPT: Federated Multi-label Prompt Tuning of Vision-Language Models
May 27, 2026Xucong Wang, Pengkun Wang, Zhe Zhao +3Multi-Label Classification VLM Adaptation
University of Science and Technology of China (USTC) · Suzhou Institute for Advanced Research, USTC · City University of Hong Kong
VidPrism: Heterogeneous Mixture of Experts for Image-to-Video Transfer
May 27, 2026Rui Lin, Chuanming Wang, Huadong MaVLM Adaptation Video Representation Learning
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, China
Rethinking Video-Language Model from the Language Input Perspective
May 27, 2026Xiang Fang, Wanlong Fang, Changshuo Wang +2VLM Adaptation Video-Language Models
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London +2
Decoupled Training with Local Reinforcement Fine-Tuning in Federated Learning
May 27, 2026Yuting Ma, Lechao Cheng, Xiaohua XuVLM Adaptation Vision-Language Alignment
School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China · School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, Anhui, China.
Reflective Dialogue between Teacher and Solver Agents for Video Question Answering
May 27, 2026Takuya Murakawa, Toru TamakiVLM Adaptation Video QA
Nagoya Institute of Technology, Nagoya, Japan
Bounded-Compute Multimodal Regression for Product-Rating Prediction
May 26, 2026William Leach, Ru He, Sizhuo Ma +4Vision-Language Models VLM Adaptation
Snap Inc.
COVD: Continual Open-Vocabulary Object Detection with Novel Concept Injection
May 26, 2026Yupeng Zhang, Ruize Han, Yuzhong Feng +3Open-Vocabulary Object Detection VLM Adaptation
Tianjin University. · Shenzhen University of Advanced Technology.
FAST-GOAL: Fast and Efficient Global-local Object Alignment Learning
May 26, 2026Hyungyu Choi, Young Kyun Jang, Chanho EomVLM Adaptation Text-Image Alignment
Department of Virtual Convergence, Graduate School of Advanced Imaging Science, Multimedia & Films (GSAIM), Chung-Ang University, Seoul, South Korea
FTibSuite: A Comprehensive Resource Suite for Tibetan Vision-Language Modeling
May 26, 2026Guixian Xu, Yide Liang, Zeli Su +5VLM Evaluation Vision-Language Models
Hainan International College, Minzu University of China · School of Information Engineering, Minzu University of China · Institute of Automation, Chinese Academy of Sciences +1
HydraPrompt: An Adaptive and Asymmetric Framework of Vision-Language Models for Synthetic Image Detection
May 26, 2026Senyuan Shi, Hao Tan, Zichang Tan +4Image Forgery Detection VLM Adaptation
Beijing University of Posts and Telecommunications · School of Advanced Interdisciplinary Sciences (SAIS), University of Chinese Academy of Sciences · MAIS, Institute of Automation, Chinese Academy of Sciences +2
Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
May 25, 2026Taha Koleilat, Hassan Rivaz, Yiming XiaoEvidential Deep Learning VLM Adaptation
Concordia University, Montreal, Canada
EVIDENT: Routing MLLM Adaptation through Entity-Grounded Visual Evidence for Cross-Domain Video Temporal Grounding
May 25, 2026Geo Ahn, Jiwook Han, Youngrae Kim +2Temporal Video Grounding VLM Robustness
Seoul National University
Closed-Loop Bidirectional Prompting for Adversarial Robustness of Vision Language Models
May 25, 2026Xiao Liu, Jiaxiang Liu, Boci Peng +6VLM Robustness Prompt Learning
University of Macau · Guangdong Institute of Intelligence Science and Technology · Peking University +4
Rethinking VLM Representation for VLA Initialization
May 25, 2026Weifeng Lin, Siyuan Huang, Hao Li +5VLM Adaptation Low-Rank Adaptation
1CUHK · 4ACE Robotics · 3Peking University +1
Addressing Exacerbated Attention Sink for Source-Free Cross-Domain Few-Shot Learning
May 25, 2026Shuai Yi, Yixiong Zou, Yuhua Li +1Source-Free Domain Adaptation Few-Shot Domain Adaptation
Huazhong University of Science and Technology
CMAP: Cross-Modal Adaptive Prompting for Multi-Domain Task-Incremental Learning
May 25, 2026Sriram MandalikaCross-Modal Learning VLM Adaptation
Hasso Plattner Institute Potsdam, Germany
MAIL++: Multi-Modal Bi-directional Agent Layer for Vision-Language Models
May 25, 2026Kaixiang Chen, Pengfei Fang, Hui XueCross-Modal Learning Cross-Modal Representation Learning
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China
Fine-Tuning Vision-Language Models for Understanding Current Damage and Scoring Priority with Quality Guard Agent
May 24, 2026Takato YasunoVLM Adaptation Structural Health Monitoring
Language Bias in LVLMs: From In-Depth Analysis to Simple and Effective Mitigation
May 24, 2026Yangneng Chen, Jing LiVision-Language Models VLM Adaptation
1Harbin Institute of Technology, Shenzhen, China.
QuoVLA: Quotient Space for Vision-Language-Action Models
May 24, 2026Xuan Wang, Yinan Wu, Haoran Duan +1Representation Learning VLM Adaptation
Department of Automation Tsinghua University
CLIP-Guided SAM: Parameter-Efficient Semantic Conditioning for Promptable Segmentation
May 24, 2026Shayan Jalilian, Abdul BaisVLM Adaptation Text-Guided Image Segmentation
University of Regina, Regina, SK, Canada
VaaWIT: Visual-Aware Adaptation of Large Language Models for Multilingual Web Image Translation
May 23, 2026Bo Li, Ronghao Chen, Ningyuan Deng +3VLM Adaptation Large Vision-Language Models
Tsinghua University Baidu Inc. Beijing, China · QuantaAlpha Beijing, China · The Hong Kong University of Science and the Technology Hong Kong, China +1
PEDESTRIANQA: A Benchmark for Vision-Language Models on Pedestrian Intention and Trajectory Prediction
May 23, 2026Naman Mishra, Shankar Gangisetty, C. V. JawaharVLM Evaluation Pedestrian Intention Prediction
CVIT, IIIT-Hyderabad, India.

May 29, 2026, cs.CVJ/K move · Enter open · S save

Variational Adapter for Cross-modal Similarity Representation

WenZhang Wei, Zhipeng Gui, Dehua Peng, Tiandi Ye+1

School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China · School of Data Science and Engineering, East China Normal University, Shanghai 200062, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

Cross-Modal Representation Learning VLM Adaptation Text-Image Alignment Cross-Modal Retrieval

Open paper AskPDF arXiv

The core of vision-language models lies in measuring cross-modal similarity within a unified representation space. However, most image-text matching or multi-class image classification datasets lack fine-grained cross-modal matching annotations, forcing the continuous similarity space into binary classification boundaries. This compression induces false negative samples and significantly impairs the generalization performance of cross-modal tasks. While prior research has attempted to mitigate this by modeling intra-modal ambiguity, it often overlooks inherent annotation flaws, leading to suboptimal uncertainty allocation. To address these challenges, we propose a Variational Adapter for Cross-modal Similarity Representation (VACSR). This approach reformulates image-text matching with fine-grained semantic scarcity as a variational inference problem. It constructs a latent space for cross-modal similarity and uses regularization techniques to mitigate overfitting to binary annotations. Experiments on image-text retrieval, domain generalization, and base-to-novel generalization demonstrate the proposed method's effectiveness and robust generalization ability.

Momentum

Related topics

Latest papers 395