VLM Adaptation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

34 papers in the last four weeks, up 209% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 395

All topics

Vision Language Model Helps Private Information De-Identification in Vision Data
Jun 8, 2026Tiejin Chen, Pingzhi Li, Kaixiong Zhou +2VLM Adaptation Privacy Protection in VLMs
Arizona State University · University of North Carolina at Chapel Hill · North Carolina State University
Failure-Aware Refinement of Vision-Language Model for Lithography Defect Detection
Jun 8, 2026Pangyun Jeong, Jiyeong Kong, Yuehua Hu +2Vision-Language Models VLM Adaptation
1HYU-KITECH Joint Department, Hanyang University, Ansan, 15588, Korea · 4Korea Institute of Industrial Technology, Ansan, 15588, Korea · 2Micro/Nano System Department, Korea University, Seoul 02841, Korea +1
Harnessing Streaming Video in the Wild
Jun 7, 2026Dingyu Yao, Shuhuan Gu, Qingyi Si +8VLM Evaluation Streaming Video Understanding
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Two Bridges, One Pathway: From VLMs to Generalizable VLAs with Embodied Trajectory-Coupled Data
Jun 7, 2026Linqi Yin, Shiduo Zhang, Shenling Qiu +11VLM Adaptation Vision-Language-Action Models
1Fudan University · 2Shanghai Innovation Institute
One Stone, Three Birds: Self-adaptive Optimal Transport for Multi-VLM Selection, Adaptation, and Ensembling
Jun 6, 2026Qiyu Xu, Zhanxuan Hu, Yu Duan +4Vision-Language Models Unsupervised Domain Adaptation
School of Computer Science and Technology and the Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · School of Information Science and Technology, Yunnan Normal University, Chenggong, Kunming, Yunnan 650500, China · School of Telecommunications Engineering, Xidian University, Xi’an 710071, China +1
Learning a Semantic Calibration Network for Open-Vocabulary Semantic Segmentation
Jun 6, 2026Yang Sun, Tao Wang, Anastasia Ioannou +1VLM Adaptation Open-Vocabulary Semantic Segmentation
College of Computer and Data Science, Fuzhou University, Fuzhou, China. · School of Computer and Big Data, Minjiang University, Fuzhou, China. · Department of Computer Science and Engineering, European University Cyprus, Nicosia, Cyprus.
Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline
Jun 6, 2026Zekai Zhang, Qinghui Chen, Maomao Xiong +6VLM Adaptation Industrial Anomaly Detection
School of Control Science and Engineering, Shandong University, Jinan, China · National Supercomputing Center in Jinan, Qilu University of Technology, Jinan, China · College of Intelligence and Computing, Tianjin University, Tianjin, China +2
Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
Jun 6, 2026Zekai Zhang, Jinglin Zhang, Qinghui Chen +8Open-Vocabulary Object Detection VLM Adaptation
School of Control Science and Engineering, Shandong University, Jinan 250061, China · Shandong Computer Science Center, Qilu University of Technology, Jinan, China · CEREMADE, University Paris Dauphine, PSL Research University, CNRS, UMR 7534, 75775 Paris, France +4
GP-Adapter: Gaussian Process CLIP-Adapter for Few-Shot Out-of-Distribution Detection
Jun 5, 2026Taisei Saito, Koretaka Ogata, Takafumi HiroiVLM Adaptation Few-Shot Image Classification
Ricoh Company, Ltd. Ebina, Japan
CL-CLIP: CLIP-Based Continual Learning Framework with Cost-Volume Category Decoupling for Object Detection
Jun 5, 2026Zihan Liu, Yuguang Yang, Shengjie Su +5VLM Adaptation Object Detection
National College for Excellent Engineers, Beihang University · School of Electronic Information Engineering, Beihang University · AI Research, Qihoo 360 +5
SS-TPT: Stability and Suitability-Guided Test-Time Prompt Tuning for Adversarially Robust Vision-Language Models
Jun 5, 2026Sunoh Kim, Daeho UmVLM Robustness VLM Adaptation
Dankook University, Yongin, South Korea · University of Seoul, Seoul, South Korea
MotionEnhancer: Leveraging Video Diffusion for Motion-Enhanced Vision-Language Models
Jun 5, 2026Yifan Xu, Chao Zhang, Ruifei Ma +4VLM Adaptation Vision-Language Alignment
School of Computer Science and Engineering, Beihang University · 2Beijing Digital Native Digital City Research Center · School of Computer Science, Peking University +1
EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Models
Jun 4, 2026Hao Wang, Qiwei Zeng, Jinghao Lin +6VLM Adaptation Medical Image Anomaly Detection
School of Computer Science, The University of Sydney, Sydney, NSW, Australia · Jilin University, Changchun, China · Northeastern University, Shenyang, China +1
DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Models
Jun 4, 2026Zhuoming Liu, Jinhong Lin, Kwan Man Cheng +3Vision-Language Models VLM Adaptation
University of Wisconsin–Madison · 2West Lafayette Jr./Sr. High School
UNIVID: Unified Vision-Language Model for Video Moderation
Jun 4, 2026Kejuan Yang, Yizhuo Zhang, Mingyuan Du +6Unified Multimodal Models VLM Adaptation
Bytedance
Geometry-Aware Distillation for Prompt Tuning Biomedical Vision-Language Models
Jun 3, 2026Tran Dinh Tien, Zhiqiang ShenVLM Adaptation VLM Distillation
Department of Machine Learning Mohamed bin Zayed University of Artificial Intelligence
VISTA: Vision-Grounded and Physics-Validated Adaptation of UMI data for VLA Training
Jun 3, 2026Siyuan Yang, Linzheng Guo, Ouyang Lu +10VLM Adaptation Robotic Data Collection
Institute of AI (TeleAI), China Telecom · University of Science and Technology of China · 4Northwestern Polytechnical University +5
Stateful Visual Encoders for Vision-Language Models
Jun 3, 2026Zirui Wang, Junwei Yu, Adam Yala +3Vision-Language Models Visual Representation Learning
1Voio, Inc. · 2UC Berkeley · 3UCSF
Frames2LoRA: Parametric Video Internalization for Vision-Language Models
Jun 3, 2026Manan Suri, Sarvesh Baskar, Dinesh ManochaVLM Adaptation Efficient VLM Inference
University of Maryland, College Park
Visual Instruction Tuning Aligns Modalities through Abstraction
Jun 2, 2026Luis Palacios, Lorenzo Basile, Diego Doimo +1VLM Adaptation VLM Interpretability
Area Science Park, Trieste, Italy
Vision-language Models for Driver Monitoring Systems: A Driver Activity Description Dataset
Jun 1, 2026David J. Lerch, Sarath Mulugurthi, Manuel Martin +2VLM Evaluation VLM Adaptation
Fraunhofer IOSB, Karlsruhe, Germany · Technische Hochschule Ingolstadt, Ingolstadt, Germany · Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Low-Rank Adaptation of Frozen Vision-Language Models for Blind Image Quality Assessment
Jun 1, 2026Bishr Omer Adam, Xu LiVLM Adaptation Low-Rank Adaptation
School of Electronics and Information, Northwestern Polytechnical University, Xi’an, China
Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigation
Jun 1, 2026Muyi Bao, Yuxin Cai, Hang Xu +7Embodied Navigation VLM Adaptation
Carnegie Mellon University · Nanyang Technological University
Ryze: Evidence-Enriched Data Synthesis from Biomedical Papers
May 30, 2026Yeqi Huang, Yue Chen, Yanwei Ye +2VLM Adaptation Synthetic Data Generation
University of Edinburgh, United Kingdom
T-CLIP: Enabling Thermal Perception for Contrastive Language-Image Pretraining
May 30, 2026Tayeba Qazi, Ayush Maheshwari, Prerana Mukherjee +1Thermal Imaging Image Captioning
Indian Institute of Technology Delhi, India · NVIDIA AI Technology Center, India · Jawaharlal Nehru University, India
Per-Group Error, Not Total MSE: Fine-Tuning Vision-Language-Action Models for 11-DoF Mobile Manipulation
May 29, 2026Pau Montagut Bofi, Mario García Blasco, Tessa Pulli +1Mobile Manipulation VLM Adaptation
Vision for Robotics Lab (V4R), TU Wien, Vienna, Austria
"Înţelegi Româneşte?'' A Recipe for Romanian Vision-Language Models
May 29, 2026Mihai Masala, Marius Leordeanu, Mihai Dascalu +1VLM Evaluation Vision-Language Models
National University of Science and Technology POLITEHNICA Bucharest, Romania
FBHM: Functional Benchmarking and Steering of VLMs for Hateful Meme Detection
May 29, 2026Paramananda Bhaskar, Naquee Rizwan, Daksh Jogchand +2VLM Evaluation VLM Adaptation
1Indian Institute of Technology (IIT), Kharagpur · 2Microsoft
Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval
May 29, 2026Alicja Dobrzeniecka, Filip Szatkowski, Sebastian Cygert +2Multimodal IR VLM Adaptation
NASK National Research Institute · IDEAS Research Institute Warsaw University of Technology · Warsaw University of Technology +2
A Pilot Study on Curator-Guided Multilingual Art Description for Blind and Low-Vision Audiences with Small Vision-Language Models
May 29, 2026Iosif Tsangko, Andreas Triantafyllopoulos, George Margetis +2Image Captioning VLM Adaptation
Technical University of Munich Munich, Germany · Foundation for Research and Technology – Hellas Heraklion, Greece · National University of Science and Technology Politehnica Bucharest Bucharest, Romania

Jun 8, 2026, cs.AIJ/K move · Enter open · S save

Vision Language Model Helps Private Information De-Identification in Vision Data

Tiejin Chen, Pingzhi Li, Kaixiong Zhou, Tianlong Chen+1

Arizona State University · University of North Carolina at Chapel Hill · North Carolina State University

VLM Adaptation Privacy Protection in VLMs

Open paper AskPDF arXiv

Visual Language Models (VLMs) have gained significant popularity due to their remarkable ability. While various methods exist to enhance privacy in text-based applications, privacy risks associated with visual inputs remain largely overlooked such as Protected Health Information (PHI) in medical images. To tackle this problem, two key tasks: accurately localizing sensitive text and processing it to ensure privacy protection should be performed. To address this issue, we introduce VisShield (Vision Privacy Shield), an end-to-end framework designed to enhance the privacy awareness of VLMs. Our framework consists of two key components: a specialized instruction-tuning dataset OPTIC (Optical Privacy Text Instruction Collection) and a tailored training methodology. The dataset provides diverse privacy-oriented prompts that guide VLMs to perform targeted Optical Character Recognition (OCR) for precise localization of sensitive text, while the training strategy ensures effective adaptation of VLMs to privacy-preserving tasks. Specifically, our approach ensures that VLMs recognize privacy-sensitive text and output precise bounding boxes for detected entities, allowing for effective masking of sensitive information. Extensive experiments demonstrate that our framework significantly outperforms existing approaches in handling private information, paving the way for privacy-preserving applications in vision-language models. Our dataset and code can be found here.

Momentum

Related topics

Latest papers 395