Visual Prompt Learning

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 167% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 78

All topics

Seeing the Invisible: Physics-Guided Visual Prompting for Temperature- and Radiation-Aware VLA Navigation
Oct 6, 2026Hojoon Son, Fan ZhangSafe Robot Navigation Vision-Language Navigation
George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0405 USA
Reprogramming Vision-Language Models via Structured Prompt Reparameterization
Sep 29, 2026Zizhao Li, Chengyi Cai, Mohammed Yaqoob Ansari +3VLM Adaptation Few-Shot Image Classification
The University of Melbourne, Melbourne, Australia
You Only Reprogram Once: Rethinking Prolonged Training for Visual Reprogramming
Sep 29, 2026Zizhao Li, Mohammed Yaqoob Ansari, Xinyu Su +3VLM Adaptation Vision Foundation Model Adaptation
The University of Melbourne · Fudan University
Gaze Prompts: Temporally Dense Human Attention for Vision-Language-Action Fine-Tuning
Sep 28, 2026Yihan Zhou, Rui Yan, Mingcong Li +4Visuomotor Policy Learning Robotic Manipulation
Department of Automation, Tsinghua University · Beijing Key Laboratory of Embodied Intelligence Systems · Institute for Embodied Intelligence and Robotics, Tsinghua University +1
PETR: Prompt Ensembling with Training-free Routing for Vision-Language Models
Sep 20, 2026Weihan Cai, Hao Tan, Xinping Gao +2Zero-Shot Learning VLM Adaptation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, and also with the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Purple Mountain Laboratories, Nanjing, China · Beijing University of Posts and Telecommunications, Beijing, China
GAPrompt++: Multi-Granular Geometry-Aware Point Cloud Prompt for 3D Vision Model
Sep 17, 2026Zixiang Ai, Zhenyu Cui, Yufei Guo +4Point Cloud Learning Vision Foundation Model Adaptation
Wangxuan Institute of Computer Technology, Peking University, Beijing 100871, China · Intelligent Science and Technology Academy of CASIC, Beijing 100041, China · Institute of Software, Chinese Academy of Sciences, Beijing 100190, China +1
TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models
Sep 15, 2026Zeyi Shao, Haowen Hua, Jiaxin Zhang +3Prompt Learning Noisy-Label Learning
Soochow University, Suzhou, China · NVIDIA, Shanghai, China · Heriot-Watt University Malaysia, Putrajaya, Malaysia +1
PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Sep 15, 2026Xuezhi Xiang, Guanghao Wu, Heqi Xiang +4Promptable Image Segmentation Anomaly Localization
Information and Communication Engineering, Harbin Engineering University, Harbin, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin, China · Department of Computer Science, University of Toronto, Toronto, ON M5S 2E4, Canada +1
Low-Rank Prompt Learning for Vision-Language Models with Fixed-Token Bases
Sep 8, 2026Tanvir Muntakim Tonoy, Sajjad Ghiasvand, Mahnoosh Alizadeh +1Vision-Language Models Prompt Learning
Department of Electrical and Computer Engineering UC Santa Barbara Santa Barbara, CA 93106, USA
Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting
Aug 12, 2026Xikai Sun, Kebin Liu, Haotian Wang +3Video Reasoning VLM Reasoning
Tsinghua University, Beijing, China · JD Logistics, Beijing, China
Evaluating Semantic and Spatial Guidance for Foundation Model Segmentation of Small-Scale PV in Remote Sensing Imagery
Aug 11, 2026Roni Blushtein-Livnon, Tal Svoray, Osher Rafaeli +4Remote Sensing Image Segmentation Promptable Image Segmentation
Department of Environmental, Geoinformatics and Urban Planning Sciences, Ben-Gurion University of the Negev, Israel · Department of Psychology, Ben-Gurion University of the Negev, Israel · Department of Geography, The Hebrew University of Jerusalem, Isreal +1
Beyond Uniform Restoration: Empowering All-in-One Restoration with Pixel-Level Multimodal Guidance
Aug 10, 2026Chunxiao Liu, Wei Liu, Anbin Xiong +1All-in-One Image Restoration Image Restoration
Xiaomi Corporation Beijing, China
ZOMP: Zeroth-Order Multi-Modal Prompt Tuning for Vision-Language Models
Aug 8, 2026Sajjad Ghiasvand, Yifan Yang, Mahnoosh Alizadeh +1VLM Adaptation Zeroth-Order Optimization
Electrical and Computer Engineering Department, UC Santa Barbara1 · Computer Science Department, UC Santa Barbara2
Adapting Vision Foundation Models with Cascaded Semantics
Aug 5, 2026Xi Xiao, Xingjian Li, Cheng Han +8Vision Foundation Model Adaptation Visual Prompt Learning
University of Alabama at Birmingham · Carnegie Mellon University · University of Missouri–Kansas City +4
UniEvo-RS: Omni-Prompt Unified Remote Sensing Segmentation with Representative Exemplar-Driven Prototype Evolution
Aug 4, 2026Kunquan Zhang, Peilang Li, Xikun Hu +4Remote Sensing Image Segmentation Test-Time Adaptation
Sun Yat-sen University · National University of Defense Technology
SeCo-SBIR: Semantically Consistent Prompt Learning for Zero-Shot Sketch-Based Image Retrieval
Aug 4, 2026Long Hoang Dang, Tuan Nguyen Huu, Nguyen Minh Hieu +1VLM Adaptation Cross-Modal Retrieval
Posts and Telecommunications Institute of Technology Ha Noi, Vietnam
USP-Mamba: Unmixing-Derived Spectral and Structural Prompting for Hyperspectral Image Super-Resolution
Aug 3, 2026Shi Chen, Jie Zhang, Yicong ZhouMamba Hyperspectral Image Super-Resolution
Department of Computer Science, University of Macau, Macau, China
Few-Shot Concept Prompt Learning for Segmentation Foundation Models via Visual Grounding
Aug 3, 2026Rahul Venkataramani, Rachana SathishMedical Imaging Foundation Models Promptable Image Segmentation
Advanced Technology Group, GE HealthCare
Mitigating Visual Degradation in MLLMs via Spatial-Spectral Visual Anchor Learning
Aug 3, 2026Qianlong Yang, Bowen Ye, Xianda Guo +4VLM Robustness Multimodal Large Language Models
China University of Petroleum (East China) Qingdao, Shandong, China · Great Wall Motor · Shanghai Jiao Tong University Shanghai, China +3
ViP-Rig: Visual-Prompted Controllable Rigging
Jul 30, 2026Zihan Qin, Mingze Sun, Yifan Mao +6Visual Prompt Learning
1Harbin Institute of Technology · 2Tsinghua University · 3Tencent +1
Visual prompt engineering for video models
Jul 28, 2026Robert Geirhos, Yuxuan Li, Thaddäus Wiedemer +7Video Reasoning Visual Prompt Learning
Google DeepMind
Rethinking Expert Training for Model Merging with Prompt Learning
Jul 27, 2026Christos Georgakilas, Aniello Panariello, Samir El Karrat Moreno +3VLM Adaptation Multi-Task Learning
Computer Vision Center, Barcelona, Spain · Universitat Aut`onoma de Barcelona, Barcelona, Spain · AImageLab, University of Modena and Reggio Emilia, Italy
Rethinking Layer-Wise Information Allocation for Vision Foundation Model Adaptation
Jul 24, 2026Yuqi Li, Xi Xiao, Yunbei Zhang +6Information Bottleneck Vision Foundation Model Adaptation
City College of New York, CUNY · New York, NY, USA · University of Alabama at Birmingham +11
DINO-VPT: Hierarchical Visual Prompt Tuning for Joint Physical-Digital Face Anti-Spoofing
Jul 23, 2026Pierre Gallin-Martel, Mika Feng, Koichi Ito +1Vision Foundation Model Adaptation Face Presentation Attack Detection
Graduate School of Information Sciences, Tohoku University, Japan
Parameter-efficient Prompt Tuning of Vision Foundation Model With Adaptive Focal Loss for Interpretable MCI Screening
Jul 16, 2026Javad Khoramdel, Farhad Hoseyni, Amirhossein NikoofardClass-Imbalanced Learning Vision Foundation Model Adaptation
APAC Research Group, Faculty of Electrical Engineering, K. N. Toosi University of Technology, Tehran, Iran
U-shaped Multi-granularity Learning for Vision-Language Models
Jul 16, 2026Biao Chen, Yunqian Yu, Xiangxu Zhao +3VLM Adaptation Visual Prompt Learning
University of Electronic Science and Technology of China · Hithink Research, Hangzhou, China
MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation
Jul 16, 2026Yinhan Zhang, Dinwei Tan, Xianghao Kong +3Video Diffusion Models Diffusion Model Fine-Tuning
HKUST · Tencent · HKUST(GZ)
ViPSAM: Visual Prompting Medical Image Segmentation Using Segment Anything Model
Jul 15, 2026San Lee, Nalee Kim, Jeong Il Yu +2Cross-Modality Medical Image Segmentation Promptable Image Segmentation
Department of Artificial Intelligence, Sungkyunkwan University, Republic of Korea · Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Republic of Korea · Department of MetaBioHealth, Sungkyunkwan University, Republic of Korea
Parameter-Efficient Vision-Language Adaptation with Continuous Metadata Conditioning for Animal Re-Identification
Jul 10, 2026Anil Osman Tur, Tonje Knutsen Sordalen, Kim Tallaksen Halvorsen +1Animal Re-Identification Distribution Shift
Department of Computer Science, University of Verona, Verona, 37134, Italy · Institute of Marine Research, Nye Flødevigveien 20, 4817 His, Norway · University of Agder, Centre for Coastal Research, Grimstad, 4604, Norway
Simon-SR: Spatially Adaptive Modulation and Visual Prompt Adaptation for Text-Reinforced Super-Resolution
Jul 10, 2026Haotong Cheng, Yuxuan Li, Zijie CuiMultimodal Fusion Visual Prompt Learning
College of Electronic Science and Engineering, Jilin University, Changchun China

Oct 6, 2026, cs.ROJ/K move · Enter open · S save

Seeing the Invisible: Physics-Guided Visual Prompting for Temperature- and Radiation-Aware VLA Navigation

Hojoon Son, Fan Zhang

George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0405 USA

Safe Robot Navigation Vision-Language Navigation Robot Safety VLMs for Robotics Visual Prompt Learning

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have become a major paradigm for Vision-and-Language Navigation (VLN). However, in safety-critical facilities, invisible risks such as radiation or temperature spikes cannot be detected by an RGB camera, and handling each risk is expensive, requiring a new encoder, new data, and model retraining. We propose Physics-Guided Visual Prompting (PG-VP), a plug-and-play multimodal perception module that instead reuses what a frozen VLA model already does well: avoiding visible obstacles. Given a proximal radiation or thermal source, PG-VP performs a physics-guided risk assessment to determine the avoidance direction and overlays a corresponding virtual obstacle that moves across consecutive frames (Dynamic Visual Prompting). The navigation policy then naturally detours around this invisible hazard. The identical virtual obstacle is used regardless of hazard type, so the visual prompting pattern remains fixed as sensors are added. When no hazard is detected, nothing is rendered, and the policy behaves exactly as it would without PG-VP. We evaluate PG-VP on OmniNav using the val-unseen splits of R2R-CE and RxR-CE, where it guides the policy toward intended low-risk actions in 84.9% and 83.2% of cases, at a cost of 6.8 and 7.9 percentage points in navigation success rate. We further test it with distinct scenarios on a real robot in the presence of actual thermal and radiation sources, all without any retraining. The real test shows that PG-VP effectively avoids these invisible hazards, improving worst-10% average trajectory safety by 63.45% and 32.59% against thermal and radiation sources, respectively.

Momentum

Related topics

Latest papers 78