VLM Adaptation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

34 papers in the last four weeks, up 209% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 395

All topics

PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails
Jul 7, 2026Mingyang Song, Luxin Xu, Haoyu Sun +3VLM Robustness VLM Adaptation
1Fudan University · 2Tongji University · 3Virtue AI +3
SAMPLe: SAM-based Optimizer for Prompt Learning in VLMs
Jul 7, 2026Hossein Rajoli, Fatemeh Lotfi, Niloufar Alipour Talemi +3Sharpness-Aware Minimization VLM Adaptation
Clemson University, Clemson SC 29634, USA · Siemens Energy (AI Lab), Orlando FL 32826, USA · University of Arizona, Tucson AZ 85721, USA
Cross-Contextual Vision-Language Adaptation with LoRA for Personalized Severe Adverse Event Detection in Clinical Wound Monitoring
Jul 6, 2026Aditi Naiknaware, Jian Sun, Aminreza Khandan +4VLM Adaptation Low-Rank Adaptation
1San Diego State University · University of California, Los Angeles · University of California, Davis
StructuredEdit: Constraint-Aware Graphic Design Editing via Differentiable Parameter Propagation
Jul 6, 2026Veeramanohar Avudaiappan, Ritwik MuraliText-Guided Image Editing VLM Adaptation
Department of Electrical and Electronics Engineering, Amrita School of Engineering, Coimbatore, Amrita Vishwa Vidyapeetham, India · Department of Computer Science and Engineering, Amrita School of Computing, Coimbatore, Amrita Vishwa Vidyapeetham, India
USE: A Unified Self-Ensembling Framework for Test-Time Prompt Tuning
Jul 4, 2026Siru Jiang, Jian Liang, Ran He +1Ensemble Learning VLM Adaptation
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences +1
Self-Specializing Vision-Language Transmon Chip Calibration in a Physics-Grounded Environment
Jul 3, 2026Animesh Tripathy, Aswanth KrishnanVLM Adaptation Tool-Using Vision-Language Agents
QpiAI India Pvt. Ltd.
AnyGroundBench: A Multi-Domain Adaptation Benchmark for Video Grounding in VLMs
Jul 2, 2026Rintaro Otsubo, Ryo Fujii, Reina Ishikawa +6VLM Evaluation Temporal Video Grounding
Keio University · Keio AI Research Center · Keio University School of Medicine +1
InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories
Jul 2, 2026Qianyu Chen, Ziteng Feng, Canran Xiao +1Continual Learning for LLMs Memory-Efficient Fine-Tuning
1Nanyang Technological University · University of Science and Technology of China · 3Shenzhen Campus of Sun Yat-sen University
Towards Real-World Ultrasound Understanding: Large Vision-Language Models from Multi-Image Examinations with Long-Form Reports
Jul 2, 2026Bingcong Yan, Chunlei Li, Jingliang Hu +3VLM Adaptation Ultrasound Imaging
1MedAI Technology (Wuxi) Co. Ltd. · 2Technical University of Munich
CPG-PAD: Concept-Informed Prompts Guided Presentation Attack Detection
Jul 1, 2026Haoyuan Zhang, Xiangyu Zhu, Li Gao +3VLM Adaptation Domain Generalization
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · China Mobile Financial Technology Co., Ltd., Beijing 100032, China
AdaBoosting Text Prompts for Vision-Language Models
Jul 1, 2026Seokhee Jin, Changhwan Sung, Sunung Mun +2Vision-Language Models Ensemble Learning
KT Corporation, Republic of Korea · Pohang University of Science and Technology (POSTECH), Republic of Korea · National AI Research Lab, Republic of Korea
Linguistic Relative Policy Optimization for Video Anomaly Reasoning
Jul 1, 2026Jiaxu Leng, Jiankang Zheng, Mengjingcheng Mo +4VLM Adaptation RL for Language Model Reasoning
School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China · Chongqing College of Artificial Intelligence, Chongqing, China
Selective Test-Time Debiasing for CLIP via Reward Gating
Jul 1, 2026Jaeho Han, Jisoo Yang, Hyeondong Woo +3Debiased ML VLM Adaptation
Department of Artificial Intelligence, Chung-Ang University
DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images
Jul 1, 2026Ke Wu, Yanan Zhang, Yingjie Gao +5Open-Vocabulary Object Detection VLM Adaptation
National College for Excellent Engineers, Beihang University, Beijing, China · School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China · School of Computer Science and Engineering, Beihang University, Beijing, China +2
Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models
Jun 30, 2026Kaiyun Yang, Ruilin Yang, Zhimin Yao +2VLM Evaluation Multimodal ICL
University of Science and Technology of China, Hefei, China · Megvii Technology Inc., Beijing, China.
Revisiting Parameter Redundancy in Vision-Language-Action Models: Insights from VLM-to-VLA Adaptation
Jun 30, 2026Fengnian Zhang, Tao Huang, Siyu Xu +2VLM Adaptation Vision-Language-Action Models
Computer Network Information Center, Chinese Academy of Sciences · University of Chinese Academy of Sciences · School of Computer Science, Shanghai Jiao Tong University +1
Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation
Jun 30, 2026Ying Chen, Jinyue Li, Kun Wang +2VLM Adaptation Text-Guided Image Segmentation
Shenzhen Research Institute, The Chinese University of Hong Kong · University of Science and Technology of China · Nanyang Technological University +1
Vision-Language-Action Models: Experimental Insights from a Real-World UR5 Platform
Jun 29, 2026Mathilde Hochedel, Marc LalondeVLM Adaptation Robotic Manipulation
Luqia Technologies 405 Ogilvy Ave., #101 Montreal, QC, Canada
Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidation
Jun 29, 2026Naeem Paeedeh, Mahardhika Pratama, Wolfgang Mayer +3VLM Adaptation Domain-Incremental Learning
School of Computer Science and IT, Adelaide University, Australia. · Faculty of Engineering and IT, University of Technology Sydney (UTS), Australia · School of Artificial Intelligence and Computer Science, Nantong University, China +1
Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents
Jun 29, 2026Yutao Sun, Yanting Miao, Hao-Xuan Ma +8Tool Use in VLMs VLM Adaptation
1Qwen Large Model Application Team, Alibaba · 2Zhejiang University · University of Waterloo +3
ScAle: Attention Head Scaling as a Minimal Adapter for Spatial Reasoning in Vision Language Models
Jun 28, 2026Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen +2Visual Spatial Reasoning VLM Adaptation
Northeastern University, Boston, USA · EmbodyX Inc., USA · Zhejiang University, China
ComMem: Complementary Memory Systems for Test-Time Adaptation of Vision-Language Models
Jun 27, 2026Guanglong Sun, Shuang Cui, Bo Lei +6Memory-Augmented VLMs Vision-Language Models
School of Life Sciences, IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China · Institute of Software Chinese Academy of Sciences, Beijing, China · 3Beijing Academy of Artificial Intelligence, Beijing, China +2
RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning
Jun 26, 2026Yelin Wang, Zijia Song, Shuo Ye +6Remote Sensing Image Understanding VLM Adaptation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Harbin Institute of Technology, Shenzhen, China · Great Bay University, Dongguan, China +1
ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval
Jun 26, 2026Siqiao Xue, Chunxue XuVLM Robustness VLM Adaptation
Improving Vision-Language-Action Model Fine-Tuning with Structured Stage and Keyframe Supervision
Jun 25, 2026Yuan Xu, Yixiang Chen, Kai Wang +5Long-Horizon Robotic Manipulation VLM Adaptation
School of Artificial Intelligence, University of Chinese Academy of Sciences. · New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences. · FiveAges.
Action ControlNet: A Lightweight Delay-Aware Adapter for Smooth Asynchronous Control in Vision-Language-Action Models
Jun 24, 2026Tiecheng Guo, Meng GuoRobotic Control Language-Conditioned Robot Manipulation
The authors are with the School of Advanced Manufacturing and Robotics, Peking University, Beijing 100871, China.
ROAD-VLA: Robust Online Adaptation via Self-Distillation for Vision-Language-Action Models
Jun 24, 2026Kejing Wang, Toan Nguyen, Minh Hoang Nguyen +2Robot Policy Adaptation VLM Adaptation
Applied Artificial Intelligence Initiative Deakin University Geelong, Australia · Air Force Research Laboratory USA
Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs
Jun 24, 2026Wenjie Zhu, Yabin Zhang, Liang Xu +3Test-Time Adaptation VLM Adaptation
The Hong Kong Polytechnic University · Harbin Institute of Technology (Shenzhen) · Shanghai Jiao Tong University +1
TACO: Towards Task-Consistent Open-Vocabulary Adaptation in Video Recognition
Jun 24, 2026Minghao Zhu, Xiao Lin, Mengxian Hu +5VLM Adaptation OOD Generalization
Tongji University, Shanghai, China
BioMedVR: Confusion-Aware Mixture-of-Prompt Experts for Biomedical Visual Reprogramming
Jun 23, 2026Jiaxiang Liu, Tianxiang Hu, Juwei Guan +5VLM Adaptation Medical VLMs
Guangdong Institute of Intelligence Science and Technology, Zhuhai, China · Zhejiang University, Hangzhou, China · Southeast University, Nanjing, China +3

Jul 7, 2026, cs.CVJ/K move · Enter open · S save

PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails

Mingyang Song, Luxin Xu, Haoyu Sun, Minzhou Pan+2

1Fudan University · 2Tongji University · 3Virtue AI+3

VLM Robustness VLM Adaptation Content Moderation

Open paper AskPDF arXiv

Image guardrails are typically trained and evaluated under a fixed safety policy, implicitly treating safety as an intrinsic property of an image. Real deployments are different: the same image may be allowed in one product, restricted in another, and newly disallowed when a policy boundary changes. We study policy-adaptive image guardrailing, where a model must decide whether an image violates the currently supplied policy and generalize to held-out policy definitions. We introduce PolicyShiftBench, a comprehensive benchmark with 2,000 policy-discriminative instances over 265 images, where each image is paired with 7.55 policy-conditioned prompts on average to test whether models adapt to the active policy rather than relying on image-level safety priors. We then propose PolicyShiftGuard, a compact policy-conditioned guardrail trained with a two-stage training recipe that combines Randomized Policy SFT (RP-SFT) with Boundary-Pair Policy Adaptation (BP-Adapt). BP-Adapt trains matched prompts for the same image and risk category using standard label supervision and a pairwise comparison loss that separates blocking policies from passing policies. Experiments show that existing VLMs and specialized guardrails remain brittle under policy shifts, while PolicyShiftGuard substantially improves policy-sensitive performance. The 7B model achieves SOTA performance of 76.9 Avg. F1 and 72.1 Avg. PSS on PolicyShiftBench, transfers well to UnSafeBench and SafeEditBench, and improves the latency-performance trade-off with a concise output format. Ablations confirm that matched pass/block boundary pairs are essential for stable policy adaptation.

Momentum

Related topics

Latest papers 395