Visual Prompt Learning

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 167% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 78

All topics

Seeing the Invisible: Physics-Guided Visual Prompting for Temperature- and Radiation-Aware VLA Navigation
Oct 6, 2026Hojoon Son, Fan ZhangSafe Robot Navigation Vision-Language Navigation
George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0405 USA
Reprogramming Vision-Language Models via Structured Prompt Reparameterization
Sep 29, 2026Zizhao Li, Chengyi Cai, Mohammed Yaqoob Ansari +3VLM Adaptation Few-Shot Image Classification
The University of Melbourne, Melbourne, Australia
You Only Reprogram Once: Rethinking Prolonged Training for Visual Reprogramming
Sep 29, 2026Zizhao Li, Mohammed Yaqoob Ansari, Xinyu Su +3VLM Adaptation Vision Foundation Model Adaptation
The University of Melbourne · Fudan University
Gaze Prompts: Temporally Dense Human Attention for Vision-Language-Action Fine-Tuning
Sep 28, 2026Yihan Zhou, Rui Yan, Mingcong Li +4Visuomotor Policy Learning Robotic Manipulation
Department of Automation, Tsinghua University · Beijing Key Laboratory of Embodied Intelligence Systems · Institute for Embodied Intelligence and Robotics, Tsinghua University +1
PETR: Prompt Ensembling with Training-free Routing for Vision-Language Models
Sep 20, 2026Weihan Cai, Hao Tan, Xinping Gao +2Zero-Shot Learning VLM Adaptation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, and also with the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Purple Mountain Laboratories, Nanjing, China · Beijing University of Posts and Telecommunications, Beijing, China
GAPrompt++: Multi-Granular Geometry-Aware Point Cloud Prompt for 3D Vision Model
Sep 17, 2026Zixiang Ai, Zhenyu Cui, Yufei Guo +4Point Cloud Learning Vision Foundation Model Adaptation
Wangxuan Institute of Computer Technology, Peking University, Beijing 100871, China · Intelligent Science and Technology Academy of CASIC, Beijing 100041, China · Institute of Software, Chinese Academy of Sciences, Beijing 100190, China +1
TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models
Sep 15, 2026Zeyi Shao, Haowen Hua, Jiaxin Zhang +3Prompt Learning Noisy-Label Learning
Soochow University, Suzhou, China · NVIDIA, Shanghai, China · Heriot-Watt University Malaysia, Putrajaya, Malaysia +1
PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Sep 15, 2026Xuezhi Xiang, Guanghao Wu, Heqi Xiang +4Promptable Image Segmentation Anomaly Localization
Information and Communication Engineering, Harbin Engineering University, Harbin, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin, China · Department of Computer Science, University of Toronto, Toronto, ON M5S 2E4, Canada +1
Low-Rank Prompt Learning for Vision-Language Models with Fixed-Token Bases
Sep 8, 2026Tanvir Muntakim Tonoy, Sajjad Ghiasvand, Mahnoosh Alizadeh +1Vision-Language Models Prompt Learning
Department of Electrical and Computer Engineering UC Santa Barbara Santa Barbara, CA 93106, USA
Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting
Aug 12, 2026Xikai Sun, Kebin Liu, Haotian Wang +3Video Reasoning VLM Reasoning
Tsinghua University, Beijing, China · JD Logistics, Beijing, China
Evaluating Semantic and Spatial Guidance for Foundation Model Segmentation of Small-Scale PV in Remote Sensing Imagery
Aug 11, 2026Roni Blushtein-Livnon, Tal Svoray, Osher Rafaeli +4Remote Sensing Image Segmentation Promptable Image Segmentation
Department of Environmental, Geoinformatics and Urban Planning Sciences, Ben-Gurion University of the Negev, Israel · Department of Psychology, Ben-Gurion University of the Negev, Israel · Department of Geography, The Hebrew University of Jerusalem, Isreal +1
Beyond Uniform Restoration: Empowering All-in-One Restoration with Pixel-Level Multimodal Guidance
Aug 10, 2026Chunxiao Liu, Wei Liu, Anbin Xiong +1All-in-One Image Restoration Image Restoration
Xiaomi Corporation Beijing, China
ZOMP: Zeroth-Order Multi-Modal Prompt Tuning for Vision-Language Models
Aug 8, 2026Sajjad Ghiasvand, Yifan Yang, Mahnoosh Alizadeh +1VLM Adaptation Zeroth-Order Optimization
Electrical and Computer Engineering Department, UC Santa Barbara1 · Computer Science Department, UC Santa Barbara2
Adapting Vision Foundation Models with Cascaded Semantics
Aug 5, 2026Xi Xiao, Xingjian Li, Cheng Han +8Vision Foundation Model Adaptation Visual Prompt Learning
University of Alabama at Birmingham · Carnegie Mellon University · University of Missouri–Kansas City +4
UniEvo-RS: Omni-Prompt Unified Remote Sensing Segmentation with Representative Exemplar-Driven Prototype Evolution
Aug 4, 2026Kunquan Zhang, Peilang Li, Xikun Hu +4Remote Sensing Image Segmentation Test-Time Adaptation
Sun Yat-sen University · National University of Defense Technology
SeCo-SBIR: Semantically Consistent Prompt Learning for Zero-Shot Sketch-Based Image Retrieval
Aug 4, 2026Long Hoang Dang, Tuan Nguyen Huu, Nguyen Minh Hieu +1VLM Adaptation Cross-Modal Retrieval
Posts and Telecommunications Institute of Technology Ha Noi, Vietnam
USP-Mamba: Unmixing-Derived Spectral and Structural Prompting for Hyperspectral Image Super-Resolution
Aug 3, 2026Shi Chen, Jie Zhang, Yicong ZhouMamba Hyperspectral Image Super-Resolution
Department of Computer Science, University of Macau, Macau, China
Few-Shot Concept Prompt Learning for Segmentation Foundation Models via Visual Grounding
Aug 3, 2026Rahul Venkataramani, Rachana SathishMedical Imaging Foundation Models Promptable Image Segmentation
Advanced Technology Group, GE HealthCare
Mitigating Visual Degradation in MLLMs via Spatial-Spectral Visual Anchor Learning
Aug 3, 2026Qianlong Yang, Bowen Ye, Xianda Guo +4VLM Robustness Multimodal Large Language Models
China University of Petroleum (East China) Qingdao, Shandong, China · Great Wall Motor · Shanghai Jiao Tong University Shanghai, China +3
ViP-Rig: Visual-Prompted Controllable Rigging
Jul 30, 2026Zihan Qin, Mingze Sun, Yifan Mao +6Visual Prompt Learning
1Harbin Institute of Technology · 2Tsinghua University · 3Tencent +1
Visual prompt engineering for video models
Jul 28, 2026Robert Geirhos, Yuxuan Li, Thaddäus Wiedemer +7Video Reasoning Visual Prompt Learning
Google DeepMind
Rethinking Expert Training for Model Merging with Prompt Learning
Jul 27, 2026Christos Georgakilas, Aniello Panariello, Samir El Karrat Moreno +3VLM Adaptation Multi-Task Learning
Computer Vision Center, Barcelona, Spain · Universitat Aut`onoma de Barcelona, Barcelona, Spain · AImageLab, University of Modena and Reggio Emilia, Italy
Rethinking Layer-Wise Information Allocation for Vision Foundation Model Adaptation
Jul 24, 2026Yuqi Li, Xi Xiao, Yunbei Zhang +6Information Bottleneck Vision Foundation Model Adaptation
City College of New York, CUNY · New York, NY, USA · University of Alabama at Birmingham +11
DINO-VPT: Hierarchical Visual Prompt Tuning for Joint Physical-Digital Face Anti-Spoofing
Jul 23, 2026Pierre Gallin-Martel, Mika Feng, Koichi Ito +1Vision Foundation Model Adaptation Face Presentation Attack Detection
Graduate School of Information Sciences, Tohoku University, Japan
Parameter-efficient Prompt Tuning of Vision Foundation Model With Adaptive Focal Loss for Interpretable MCI Screening
Jul 16, 2026Javad Khoramdel, Farhad Hoseyni, Amirhossein NikoofardClass-Imbalanced Learning Vision Foundation Model Adaptation
APAC Research Group, Faculty of Electrical Engineering, K. N. Toosi University of Technology, Tehran, Iran
U-shaped Multi-granularity Learning for Vision-Language Models
Jul 16, 2026Biao Chen, Yunqian Yu, Xiangxu Zhao +3VLM Adaptation Visual Prompt Learning
University of Electronic Science and Technology of China · Hithink Research, Hangzhou, China
MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation
Jul 16, 2026Yinhan Zhang, Dinwei Tan, Xianghao Kong +3Video Diffusion Models Diffusion Model Fine-Tuning
HKUST · Tencent · HKUST(GZ)
ViPSAM: Visual Prompting Medical Image Segmentation Using Segment Anything Model
Jul 15, 2026San Lee, Nalee Kim, Jeong Il Yu +2Cross-Modality Medical Image Segmentation Promptable Image Segmentation
Department of Artificial Intelligence, Sungkyunkwan University, Republic of Korea · Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Republic of Korea · Department of MetaBioHealth, Sungkyunkwan University, Republic of Korea
Parameter-Efficient Vision-Language Adaptation with Continuous Metadata Conditioning for Animal Re-Identification
Jul 10, 2026Anil Osman Tur, Tonje Knutsen Sordalen, Kim Tallaksen Halvorsen +1Animal Re-Identification Distribution Shift
Department of Computer Science, University of Verona, Verona, 37134, Italy · Institute of Marine Research, Nye Flødevigveien 20, 4817 His, Norway · University of Agder, Centre for Coastal Research, Grimstad, 4604, Norway
Simon-SR: Spatially Adaptive Modulation and Visual Prompt Adaptation for Text-Reinforced Super-Resolution
Jul 10, 2026Haotong Cheng, Yuxuan Li, Zijie CuiMultimodal Fusion Visual Prompt Learning
College of Electronic Science and Engineering, Jilin University, Changchun China
Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition
Jul 7, 2026Xinda Liu, Qinyu Zhang, Weiqing Min +2VLM Adaptation Fine-Grained Image Classification
School of Information Science and Technology, Northwest University, Xi’an, Shaanxi, China · Laboratory of Intelligent Information Processing, Institute of Computing Technology, Beijing, China
SAMPLe: SAM-based Optimizer for Prompt Learning in VLMs
Jul 7, 2026Hossein Rajoli, Fatemeh Lotfi, Niloufar Alipour Talemi +3Sharpness-Aware Minimization VLM Adaptation
Clemson University, Clemson SC 29634, USA · Siemens Energy (AI Lab), Orlando FL 32826, USA · University of Arizona, Tucson AZ 85721, USA
Learning Probabilistic Prompt for Continual Learning
Jul 6, 2026Hyekang Park, Sanghoon Lee, Geon Lee +2Class-Incremental Learning Prompt Learning
1Yonsei University, Seoul, South Korea · 2Samsung Electronics, Suwon, South Korea
Hidden-Shot: Towards One-Shot Task Generalization for Low-Level Vision Generalist Models
Jul 1, 2026Shao-Jun Xia, Xianzheng Ma, Zichong MengVision Foundation Model Adaptation Few-Shot Learning
Pratt School of Engineering, Duke University, Durham, NC, 27708, USA · Department of Engineering Science, University of Oxford, Oxford, Parks Road, OX1 3PJ, UK · Khoury College of Computer Science, Northeastern University, Boston, MA, 02115, USA
CPG-PAD: Concept-Informed Prompts Guided Presentation Attack Detection
Jul 1, 2026Haoyuan Zhang, Xiangyu Zhu, Li Gao +3VLM Adaptation Domain Generalization
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · China Mobile Financial Technology Co., Ltd., Beijing 100032, China
Learning from Reliable Latent Prompts for Visual Recognition with Missing Modalities
Jun 29, 2026Taixi Chen, Nancy GuoMissing-Modality Learning Prompt Learning
School of Computing State University of New York at Binghamton
Layer-Specific Prompt Fusion Discovery via Differentiable Search in Vision Foundation Models
Jun 24, 2026Xi Xiao, Xingjian Li, Yunbei Zhang +7Vision Transformer Vision Foundation Model Adaptation
UAB · CMU · Tulane +3
Decoupling Semantics and Geometric Grounding: Spatial Visual Prompts for Language-Conditioned Imitation Learning
Jun 24, 2026Yanzhe Tang, Xinyu Shao, Yuxuan Hu +6Language-Conditioned Robot Manipulation Vision-Language Grounding
1Tsinghua University, Shenzhen, China. · 2Huawei Technologies Co., Ltd., Shenzhen, China. · 3National University of Singapore, Singapore.
BioMedVR: Confusion-Aware Mixture-of-Prompt Experts for Biomedical Visual Reprogramming
Jun 23, 2026Jiaxiang Liu, Tianxiang Hu, Juwei Guan +5VLM Adaptation Medical VLMs
Guangdong Institute of Intelligence Science and Technology, Zhuhai, China · Zhejiang University, Hangzhou, China · Southeast University, Nanjing, China +3
BoxCtrl: 3D-Aware Visual Prompting for Geometric Image Editing
Jun 22, 2026Feifei Wang, Shiyuan Yang, Xiaoyu Li +1Image Editing Visual Prompt Learning
City University of Hong Kong, China · Tencent, China
Bridging Day and Night: Unsupervised Cross-Domain Re-Identification with Synergistic Prompt and Prototype Learning
Jun 10, 2026Jiyang Xu, Rui Liu, Hang DaiUnsupervised Domain Adaptation Prototype Learning
School of Computer Science, Wuhan University, Wuhan, China
Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training
Jun 10, 2026Michal Chudoba, Sergey Alyaev, Petra Galuscakova +1LLM Fine-Tuning Multimodal Large Language Models
University of Stavanger · Stavanger, Norway · NORCE Research +1
Visual Prompting Meets Feature Reconstruction-Based Anomaly Detection with Dual-Teacher Supervision
Jun 8, 2026Mateo Diaz-Bone, Daniel Caraballo, Florian Scheidegger +11Anomaly Localization Reconstruction-Based Anomaly Detection
IBM Research Europe Zurich, Switzerland
Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline
Jun 6, 2026Zekai Zhang, Qinghui Chen, Maomao Xiong +6VLM Adaptation Industrial Anomaly Detection
School of Control Science and Engineering, Shandong University, Jinan, China · National Supercomputing Center in Jinan, Qilu University of Technology, Jinan, China · College of Intelligence and Computing, Tianjin University, Tianjin, China +2
Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
Jun 6, 2026Zekai Zhang, Jinglin Zhang, Qinghui Chen +8Open-Vocabulary Object Detection VLM Adaptation
School of Control Science and Engineering, Shandong University, Jinan 250061, China · Shandong Computer Science Center, Qilu University of Technology, Jinan, China · CEREMADE, University Paris Dauphine, PSL Research University, CNRS, UMR 7534, 75775 Paris, France +4
Geometry-Aware Distillation for Prompt Tuning Biomedical Vision-Language Models
Jun 3, 2026Tran Dinh Tien, Zhiqiang ShenVLM Adaptation VLM Distillation
Department of Machine Learning Mohamed bin Zayed University of Artificial Intelligence
Imagine Before You Draw: Visual Prompt Engineering for Image Generation
Jun 3, 2026Liyu Jia, Fengda Zhang, Jiachun Pan +7Conditional Image Generation T2I Generation
1Nanyang Technological University · 2National University of Singapore · 3Zhejiang University +1
Beyond Low-Rank: Low-Rank Sparse Prompting via Spiking Neural Network and Prompt Factorization
Jun 1, 2026Yumiao Zhao, Bo Jiang, Beibei Wang +3Vision Foundation Model Adaptation Low-Rank Adaptation
Information Materials and Intelligent Sensing Laboratory of Anhui Province, Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University
Personalize Your Large Vision-language Models With In-context Prompt Tuning
May 29, 2026Yanshu Li, Jiaqian Li, Kuai Yu +4Vision-Language Models Large Vision-Language Models
Brown University · Columbia University · University of Alabama at Birmingham +2
BadBone: Backdoor Attacks Against Backbone Models in Visual Prompt Learning
May 29, 2026Ziqing Yang, Rui Wen, Xinlei He +3Adversarial Attacks Backdoor Attacks
1CISPA Helmholtz Center for Information Security · Institute of Science Tokyo · 3Wuhan University +1
FedMPT: Federated Multi-label Prompt Tuning of Vision-Language Models
May 27, 2026Xucong Wang, Pengkun Wang, Zhe Zhao +3Multi-Label Classification VLM Adaptation
University of Science and Technology of China (USTC) · Suzhou Institute for Advanced Research, USTC · City University of Hong Kong
LV-OSD: Language-Vision-Complementary Open-Set Object Detection
May 27, 2026Yupeng Zhang, Ruize Han, Wei Feng +2Open-Vocabulary Object Detection Object Detection
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China · Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen 518107, Guangdong, China
Leveraging Text-to-Image Diffusion Models for Unsupervised Visual Object Tracking
May 26, 2026Zhengbo Zhang, Zhigang Tu, Junsong Yuan +2Visual Object Tracking Visual Prompt Learning
Information Systems Technology and Design Pillar, Singapore University of Technology and Design, Singapore · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, China · Department of Computer Science and Engineering, University at Buffalo, State University of New York, USA +1
HydraPrompt: An Adaptive and Asymmetric Framework of Vision-Language Models for Synthetic Image Detection
May 26, 2026Senyuan Shi, Hao Tan, Zichang Tan +4Image Forgery Detection VLM Adaptation
Beijing University of Posts and Telecommunications · School of Advanced Interdisciplinary Sciences (SAIS), University of Chinese Academy of Sciences · MAIS, Institute of Automation, Chinese Academy of Sciences +2
Image-Conditioned Instance Prompt Network for Referring Remote Sensing Image Segmentation
May 23, 2026Biaoyu Ren, Qingsheng Wang, Cun Xu +2Remote Sensing Image Segmentation Referring Image Segmentation
School of Computer Science, Northwestern Polytechnical University, Xi'an, China · School of Computer Science, Northwestern Polytechnical University, Xi’an, China · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China +1
LaCoVL-FER: Landmark-Guided Contrastive Learning Network with Vision-Language Enhancement for Facial Expression Recognition
May 19, 2026Jiaxin Wang, Muwei Jian, Hui Yu +2VLM Adaptation Facial Expression Recognition
School of Airspace Science and Engineering, Shandong University, Weihai, China · School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China · School of Psychology and Neuroscience, University of Glasgow, Glasgow, U.K. +1
Dual-Prompt CLIP with Hybrid Visual Encoders for Occluded Person Re-Identification
May 19, 2026Zhangjian Ji, Shaotong Qiao, Kai Feng +1Visual Prompt Learning Person Re-Identification
School of Computer & Information Technology, Shanxi University, Wucheng Rd.92, Taiyuan, 030006, Shanxi, China · Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Wucheng Rd.92, Taiyuan, 030006, Shanxi, China
Exposing Functional Fusion: A New Class of Strategic Backdoor in Dynamic Prompt Architectures
May 19, 2026Zeyao Liu, Zhendong Zhao, Xiaojun Chen +3Adversarial Attacks Vision Transformer
Institute of Information Engineering, Chinese Academy of Sciences · State Key Laboratory of Cyberspace Security Defense · School of Cyber Security, University of Chinese Academy of Sciences +1
WOW-Seg: A Word-free Open World Segmentation Model
May 16, 2026Danyang Li, Tianhao Wu, Bin Li +5Image Segmentation Open-Set Recognition
2VCIP, CS, Nankai University · 1NKIARI, Shenzhen Futian · 3AAIS, Nankai University +2
Cluster-Aware Neural Collapse Prompt Tuning for Long-Tailed Generalization of Vision-Language Models
May 12, 2026Boyang Guo, Liang Li, Lin Peng +3Long-Tail Learning VLM Adaptation
Hangzhou Dianzi University, Hangzhou, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · The Hong Kong Polytechnic University, Hong Kong, China +2

Oct 6, 2026, cs.ROJ/K move · Enter open · S save

Seeing the Invisible: Physics-Guided Visual Prompting for Temperature- and Radiation-Aware VLA Navigation

Hojoon Son, Fan Zhang

George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0405 USA

Safe Robot Navigation Vision-Language Navigation Robot Safety VLMs for Robotics Visual Prompt Learning

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have become a major paradigm for Vision-and-Language Navigation (VLN). However, in safety-critical facilities, invisible risks such as radiation or temperature spikes cannot be detected by an RGB camera, and handling each risk is expensive, requiring a new encoder, new data, and model retraining. We propose Physics-Guided Visual Prompting (PG-VP), a plug-and-play multimodal perception module that instead reuses what a frozen VLA model already does well: avoiding visible obstacles. Given a proximal radiation or thermal source, PG-VP performs a physics-guided risk assessment to determine the avoidance direction and overlays a corresponding virtual obstacle that moves across consecutive frames (Dynamic Visual Prompting). The navigation policy then naturally detours around this invisible hazard. The identical virtual obstacle is used regardless of hazard type, so the visual prompting pattern remains fixed as sensors are added. When no hazard is detected, nothing is rendered, and the policy behaves exactly as it would without PG-VP. We evaluate PG-VP on OmniNav using the val-unseen splits of R2R-CE and RxR-CE, where it guides the policy toward intended low-risk actions in 84.9% and 83.2% of cases, at a cost of 6.8 and 7.9 percentage points in navigation success rate. We further test it with distinct scenarios on a real robot in the presence of actual thermal and radiation sources, all without any retraining. The real test shows that PG-VP effectively avoids these invisible hazards, improving worst-10% average trajectory safety by 63.45% and 32.59% against thermal and radiation sources, respectively.

Momentum

Related topics

Latest papers 78