VLM Adaptation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

34 papers in the last four weeks, up 209% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 395

All topics

0.5%>100%: Bidirectional Reciprocal Learning for Referring Image Segmentation
Sep 21, 2026Xiaoqiang Lu, Licheng Jiao, Lingling Li +5VLM Adaptation Vision Foundation Model Adaptation
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, International Research Center for Intelligent Perception and Computation, School of Artificial Intelligence, Xidian University, Xi’an, Shaanxi Province, 710071, China
Hierarchical Prompt Learning for Hyperbolic Vision-Language Models
Sep 21, 2026Andro Erdelez, Pascal Mettes, Behzad BozorgtabarVision-Language Models Prompt Learning
Aarhus University · University of Amsterdam
Reassessing Global Gradient-Norm Imbalance in BLIP Fine-Tuning Across Physical Domains
Sep 20, 2026Kiran Naseer, Samreen Azhar, Dwarikanath MahapatraImage Captioning VLM Adaptation
Department of Computer Science, University of Gujrat, Pakistan · Department of Computer Science, Khalifa University, Abu Dhabi, UAE
PETR: Prompt Ensembling with Training-free Routing for Vision-Language Models
Sep 20, 2026Weihan Cai, Hao Tan, Xinping Gao +2Zero-Shot Learning VLM Adaptation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, and also with the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Purple Mountain Laboratories, Nanjing, China · Beijing University of Posts and Telecommunications, Beijing, China
Bridging the Perceptual Gap: Residual-Enhanced Downscaling and Manifold-Aware Perception Alignment Adaptation for NR-IQA
Sep 15, 2026Yu Li, Zhengran Shen, Yachun Mi +2VLM Adaptation Cross-Modal Alignment
Harbin Institute of Technology, Harbin, China.
VPRef: A Cross-Domain Benchmark for Referring Remote Sensing Image Segmentation
Sep 15, 2026Quanwei Liu, Tao Huang, Jiaqi Yang +1Remote Sensing Image Segmentation VLM Adaptation
College of Science and Engineering, James Cook University, Cairns, 4878, Australia
Option-Aware Retrieval and Task-Specific VLM Adaptation for Medical VQA
Sep 14, 2026Tristan Kirscher, Niklas C. Koser, Soren PirkVLM Adaptation Multiple-Choice Question Answering
ICube, Institut Strauss · ICube Laboratory, CNRS UMR 7357, University of Strasbourg, Strasbourg, France · CLCC Institut Strauss, Strasbourg, France +2
Planning in the Backbone: DiffAdapterVLA for Native Continuous Trajectory Generation with Driving VLMs
Sep 14, 2026Changxin Lu, Xiaoliang Meng, Yu Wu +5Trajectory Generation VLMs for Autonomous Driving
School of Remote Sensing and Information Engineering, Wuhan University · Dongfeng Research & Development Institute
Pre-PEFT Probing: Weight Statistics and Perturbation Robustness for Layer Selection in VLM Vision Encoders
Sep 14, 2026Qingtao Xia, Jiahua Bao, Siyao Cheng +1VLM Adaptation Parameter-Efficient Fine-Tuning
Faculty of Computing Harbin Institute of Technology Harbin, China
Your Model Already Knows Don't Teach It, Learn to Ask It: Soft Prompting for Few-Shot Adaptation of Vision-Language Models
Sep 12, 2026Gautam Rajendrakumar Gare, Siyi Li, Hewei Wang +5Few-Shot Domain Adaptation VLM Adaptation
Carnegie Mellon University · Apple
Learning to Adapt and Calibrate: Score Distribution Alignment for Few-Shot Uncertainty Prediction in Medical VLMs
Sep 9, 2026Xuan Cuong Ngo, Ngan LeVLM Adaptation Conformal Prediction
University of Arkansas Fayetteville, Arkansas, USA
On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Data
Sep 9, 2026Hongyuan Zhang, Xianda Guo, Yanlun Peng +6VLM Adaptation VLM Distillation
The University of Hong Kong, Hong Kong SAR, China · Great Wall Motor, China · School of Information Science and Technology, University of Science and Technology of China, China +4
Low-Rank Prompt Learning for Vision-Language Models with Fixed-Token Bases
Sep 8, 2026Tanvir Muntakim Tonoy, Sajjad Ghiasvand, Mahnoosh Alizadeh +1Vision-Language Models Prompt Learning
Department of Electrical and Computer Engineering UC Santa Barbara Santa Barbara, CA 93106, USA
To Adapt or Not to Adapt? Selective Adaptation for Vision-Language Models
Sep 8, 2026Siru Jiang, Yuwei Liang, Jian Liang +2Test-Time Adaptation VLM Adaptation
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, China · NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, China +1
Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection
Sep 7, 2026Manwen Yang, Leqian Ding, Yu Guo +1VLM Adaptation Anomaly Localization
School of Software Engineering, Xi’an Jiaotong University · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
Lightweight Adaptation of General-Purpose VLMs for Multispectral and SAR Image Understanding
Sep 2, 2026Shanji Liu, Kelu Yao, Junxiao Xue +5VLM Adaptation Remote Sensing Image Understanding
Zhejiang University · Zhejiang Lab
Text Capability Loss in Vision-Language Adaptation: An Attention-Sink Diagnosis
Sep 1, 2026Minsik Choi, Geewook Kim, Young Geun KimVision-Language Models VLM Adaptation
Korea University · NAVER Cloud AI
Teaching Vision-Language Models to Use the Scale They Are Given: Label-Free Equivariance Training for Metric Physical Reasoning
Sep 1, 2026Kaizhen Tan, Yang Feng, Heqing Du +3Physical Reasoning VLM Adaptation
New York University · Columbia University · Carnegie Mellon University
CoLT-Drive: Counterfactual Long-Tail Benchmarking and Knowledge-Preserving Adaptation for Driving Affordance Prediction
Aug 31, 2026Zhengxu Tang, Guofeng Cui, Ziyu Gong +8Autonomous Driving Benchmarks VLMs for Autonomous Driving
NVIDIA
LiST: Local-Simplex Test-Time LoRA Fusion
Aug 23, 2026Yihua Shao, Jia Li, Siyu Chen +12VLM Adaptation Test-Time Adaptation of VLMs
PolyU · CASIA · GDUT +7
Dual Anchors, Do It Better: Hierarchical Group Merging for Zero-Shot Anomaly Detection
Aug 12, 2026Jimin Roh, DongKyu Kim, Suk-Ju KangVLM Adaptation Multimodal Anomaly Detection
Sogang University · LG Electronics · NAVER Cloud
When Visual Signals Mislead: A Mechanistic Study of Attribute Hallucination in Vision-Language Models
Aug 11, 2026Yufei Zhang, Chenlu Zhan, Hongwei WangHallucination Detection in VLMs VLM Adaptation
Zhejiang University
Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models
Aug 11, 2026Yuhang Song, Bor-Jiun Lin, Jiaxu Liu +3Memory-Augmented VLMs Vision-Language Models
Department of Computer Science, University of Liverpool · Department of Computer Science, National Tsinghua University · Imperial College London +1
DistMoE: Private-data Rehearsal-free Routing in Mixture-of-Experts for Distributed Instruction Tuning
Aug 10, 2026Mainak Singha, Niccolò Biondi, Elisa Ricci +1Mixture-of-Experts Language Models VLM Adaptation
University of Trento, Italy · Fondazione Bruno Kessler, Italy · University of Bergamo, Italy
Real Data Closes Synthetic-to-Real Gap in Optical Chemical Structure Recognition
Aug 10, 2026Yani Guan, Dengpan Dong, Zi Wei +4Synthetic-to-Real Domain Adaptation Vision-Language Models
SES AI Corporation
SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs
Aug 10, 2026Shiwei Gan, Xiao Liu, Yafeng Yin +5Sign Language Translation VLM Adaptation
University of Warwick Coventry, United Kingdom
ZOMP: Zeroth-Order Multi-Modal Prompt Tuning for Vision-Language Models
Aug 8, 2026Sajjad Ghiasvand, Yifan Yang, Mahnoosh Alizadeh +1VLM Adaptation Zeroth-Order Optimization
Electrical and Computer Engineering Department, UC Santa Barbara1 · Computer Science Department, UC Santa Barbara2
MuST-VAD: Mutual Structured Learning for Video Anomaly Detection
Aug 7, 2026Satoshi Hashimoto, Hitoshi Nishimura, Mori KurokawaVLM Adaptation Weakly Supervised Video Anomaly Detection
KDDI Research, Inc. Fujimino, Saitama, Japan
When Semantics Saturate or Emerge: Adaptation-Conditional Semantic Utility in Source-Free Cross-Domain Few-Shot Learning
Aug 7, 2026Wei Liu, Xing Deng, Haijian ShaoFew-Shot Domain Adaptation VLM Adaptation
College of Computer Science, Jiangsu University of Science and Technology, Zhenjiang 212003, China
Test-Time Adaptation with Online Personalized Energy-Based Cache for Fine-Grained Video Expression Recognition
Aug 6, 2026Masoumeh Sharafi, Muhammad Osama Zeeshan, Soufiane Belharbi +3Test-Time Adaptation VLM Adaptation
LIVIA, Department of Systems Engineering, École de technologie supérieure, Montreal, Canada · LIVIA, Department of Software and IT Engineering, École de technologie supérieure, Montreal, Canada

Sep 21, 2026, cs.CVJ/K move · Enter open · S save

0.5%>100%: Bidirectional Reciprocal Learning for Referring Image Segmentation

Xiaoqiang Lu, Licheng Jiao, Lingling Li, Yuting Yang+4

Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, International Research Center for Intelligent Perception and Computation, School of Artificial Intelligence, Xidian University, Xi’an, Shaanxi Province, 710071, China

VLM Adaptation Vision Foundation Model Adaptation Referring Image Segmentation Vision-Language Alignment Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Recent advances in vision foundation models (VFMs) have shown remarkable capabilities across diverse unimodal visual tasks. However, adapting VFMs to referring image segmentation (RIS) typically necessitates precise vision-language alignment via full fine-tuning, incurring substantial computational overhead and risking catastrophic forgetting. While existing parameter-efficient fine-tuning (PEFT) methods enable safe knowledge transfer with minimal training costs, they predominantly operate independently within individual modalities or focus exclusively on unidirectional guidance from language to vision, overlooking progressive cross-modal interaction and visual feedback for textual refinement. To address these limitations, we propose

\textbf{B}

idirectional

\textbf{R}

eciprocal

\textbf{L}

earning (

\textbf{BRL}

), a novel adapter-based PEFT framework that facilitates hierarchical, bidirectional information flow within both token-mixing and channel-mixing layers of frozen foundation models. Specifically, BRL introduces two complementary lightweight modules. The Reciprocal Attention Adapter (RAA) performs cross-modal query-key exchanges at the token level, enabling visual and linguistic tokens to mutually attend to each other for fine-grained spatial grounding. The Reciprocal Gate Adapter (RGA) generates cross-modal gating signals at the channel level, allowing global semantic context from one modality to adaptively recalibrate channel activations of the other. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg benchmarks demonstrate the superiority of BRL over prior RIS methods, achieving state-of-the-art performance while requiring less than 0.5% backbone parameter updates. Code and models will be released at https://github.com/xiaoqiang-lu/BRL.

Momentum

Related topics

Latest papers 395