Visual Prompt Learning

Ask about this topic

Overview

Visual prompt learning studies how to steer pretrained models toward new tasks by changing prompts rather than extensively retraining the models. Approaches include soft tokens inserted into model layers, prompt pools with dynamic routing, and visual cues such as masks, points, overlays, or selected frames; some methods also align visual prompts with text. These techniques are used for tasks including classification, segmentation, detection, and multimodal reasoning, often with frozen backbones and little task-specific data. A central challenge is adapting to new domains and noisy inputs while preserving the pretrained model’s generalization and keeping prompts efficient.

Latest papers

CardsList

123 papers7 papers in the last four weeks, down 36% on the four weeks before. 0.0% of all new papers.

  1. Visual prompting reimagined: The power of the Activation Prompts

    Apr 7, 2026Yihua Zhang, Hongkang Li, Yuguang Yao +5Visual Prompt LearningVision Foundation Model Adaptation

    Michigan State University · University of Pennsylvania · New Jersey Institute of Technology +2

  2. Love Me, Love My Label: Rethinking the Role of Labels in Prompt Retrieval for Visual In-Context Learning

    Apr 4, 2026Tianci Luo, Haohao Pan, Jinpeng Wang +5In-Context LearningRetrieval-Augmented ICL

    Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Computer Science and Engineering, Northeastern University · Harbin Institute of Technology, Shenzhen

  3. UniSpector: Towards Universal Open-set Defect Recognition via Spectral-Contrastive Visual Prompting

    Apr 3, 2026Geonuk Kim, Minhoi Kim, Kangil Lee +5Industrial Anomaly DetectionIndustrial Inspection

    LG Energy Solution

  4. Unlocking Positive Transfer in Incrementally Learning Surgical Instruments: A Self-reflection Hierarchical Prompt Framework

    Apr 3, 2026Yu Zhu, Kang Li, Zheng Li +1Class-Incremental LearningSurgical Video Understanding

    The Chinese University of Hong Kong · University of Electronic Science and Technology of China

  5. PET-DINO: Unifying Visual Cues into Grounding DINO with Prompt-Enriched Training

    Apr 1, 2026Weifu Fu, Jinyang Li, Bin-Bin Gao +6Object DetectionOpen-Vocabulary Object Detection

    YouTu Lab, Tencent · Huazhong University of Science and Technology · Kling Team, Kuaishou Technology

  6. Domain-Invariant Prompt Learning for Vision-Language Models

    Mar 30, 2026Arsham Gholamzadeh Khoee, Yinan Yu, Robert FeldtVLM AdaptationVisual Prompt Learning

    Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden

  7. IP-SAM: Rethinking Prompt-Conditioned Segmentation for Prompt-Absent Deployment

    Mar 28, 2026Huiyao Zhang, Jin Bai, Rui Guo +3Image SegmentationPromptable Image Segmentation

    University of Chinese Academy of Sciences · Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences

  8. ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting

    Mar 24, 2026Yeonkyung Lee, Dayun Ju, Youngmin Kim +2Temporal Video UnderstandingVideo-Language Models

    Yonsei University

  9. SOUPLE: Enhancing Audio-Visual Localization and Segmentation with Learnable Prompt Contexts

    Mar 24, 2026Khanh Binh Nguyen, Chae Jung ParkAudio-Visual UnderstandingVisual Prompt Learning

    Deakin University, Australia · National Cancer Center, South Korea

  10. Parameter-efficient Prompt Tuning and Hierarchical Textual Guidance for Few-shot Whole Slide Image Classification

    Mar 23, 2026Jayanie Bogahawatte, Sachith Seneviratne, Saman HalgamugeWhole-Slide Image AnalysisParameter-Efficient Fine-Tuning

    AI, Optimization and Pattern Recognition Research Group, Dept. of Mechanical Eng., University of Melbourne, Australia

  11. ALADIN:Attribute-Language Distillation Network for Person Re-Identification

    Mar 23, 2026Wang Zhou, Boran Duan, Haojun Ai +2Person Re-IdentificationCross-Modal Knowledge Distillation

    Wuhan University No.299 Bayi Road, 430072, PR China

  12. Dual Prompt-Driven Feature Encoding for Nighttime UAV Tracking

    Mar 20, 2026Yiheng Wang, Changhong Fu, Liangliang Yao +2Visual Object TrackingVisual Prompt Learning

    Pratt School of Engineering, Duke University, Durham 27705, United States · School of Mechanical Engineering, Tongji University, Shanghai 201804, China · Shanghai Key Laboratory of Wearable Robotics and Human Machine Interaction, Tongji University, Shanghai 201804, China +1

  13. DiffVP: Differential Visual Semantic Prompting for LLM-Based CT Report Generation

    Mar 18, 2026Yuhe Tian, Kun Zhang, Haoran Ma +4CT Report GenerationRadiology Report Generation

    Department of Electronic Engineering and Information Science, School of Information Science and Technology, University of Science and Technology of China (USTC), Hefei, Anhui 230026, China · Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advanced Research, University of Science and Technology of China (USTC), Suzhou, Jiangsu 215123, China · Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, University of Science and Technology of China (USTC), Suzhou, Jiangsu 215123, China +2

  14. VirPro: Visual-referred Probabilistic Prompt Learning for Weakly-Supervised Monocular 3D Detection

    Mar 18, 2026Chupeng Liu, Jiyong Rao, Shangquan Sun +23D Object DetectionMonocular 3D Object Detection

    The University of Sydney · Tongji University · Nanyang Technological University

  15. Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing

    Mar 16, 2026Jiahe Song, Chuang Wang, Yinfan Wang +11Visual Prompt LearningReinforcement Learning with Verifiable Rewards

    Shanghai Jiao Tong University · Shanghai AI Laboratory · Beihang University +2

  16. VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Prompting

    Mar 15, 2026Daeun Lee, Shoubin Yu, Yue Zhang +1RL for Video ReasoningVideo Reasoning

    University of North Carolina, Chapel Hill, USA

  17. FedBPrompt: Federated Domain Generalization Person Re-Identification via Body Distribution Aware Visual Prompts

    Mar 13, 2026Xin Xu, Weilong Li, Wei Liu +5Federated Domain GeneralizationPerson Re-Identification

    School of Computer Science and Technology, Wuhan University of Science and Technology, China · Nanyang Technological University, Singapore · NERC for Multimedia Software, School of Computer Science, Wuhan University, China +3

  18. Noise-aware few-shot learning through bi-directional multi-view prompt alignment

    Mar 12, 2026Lu Niu, Cheng XueFew-Shot LearningVisual Prompt Learning

    Southeast University · AIIA, Ministry of Education, China

  19. Guided Prompt Evolution for Vision-Language Models Adaptation

    Mar 10, 2026Enming Zhang, Jiayang Li, Yanlong Wang +3VLM AdaptationFew-Shot Learning

    Tsinghua Shenzhen International Graduate School, Tsinghua University · Sun Yat-sen University · Chinese University of Hong Kong, Shenzhen

  20. Local-Global Prompt Learning via Sparse Optimal Transport

    Mar 9, 2026Deniz Kizaroğlu, Ülku Tuncer Küçüktas, Emre Çakmakyurdu +1VLM AdaptationVisual Prompt Learning

    Graduate School of Informatics, Middle East Technical University, Ankara, Turkey · Department of Electrical and Electronics Engineering, Gazi University, Ankara, Turkey

  21. Whispering to a Blackbox: Bootstrapping Frozen OCR with Visual Prompts

    Mar 5, 2026Samandar Samandarov, Nazirjon Ismoiljonov, Abdullah Sattorov +1Visual Prompt LearningOptical Character Recognition

    Duke University

  22. Beyond Prompt Degradation: Prototype-guided Dual-pool Prompting for Incremental Object Detection

    Mar 2, 2026Yaoteng Zhang, Zhou Qing, Junyu Gao +1Object DetectionClass-Incremental Learning

    Northwestern Polytechnical University

  23. Graph-of-Mark: Promote Spatial Reasoning in Multimodal Language Models with Graph-Based Visual Prompting

    Mar 2, 2026Giacomo Frisoni, Lorenzo Molfetta, Mattia Buzzoni +1Spatial Reasoning in Language ModelsVLM Reasoning

    Department of Computer Science and Engineering, University of Bologna

  24. PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Prompts

    Mar 2, 2026Xianqi Wang, Hao Yang, Hangtian Wang +4Stereo MatchingVisual Prompt Learning

    Huazhong University of Science and Technology · Optics Valley Laboratory

  25. Evaluating Visual Prompts with Eye-Tracking Data for MLLM-Based Human Activity Recognition

    Feb 27, 2026Jae Young Choi, Seon Gyeom Kim, Hyungjun Yoon +7Human Activity RecognitionMultimodal Large Language Models

    KAIST · Adobe Research

  26. pMoE: Prompting Diverse Experts Together Wins More in Visual Adaptation

    Feb 26, 2026Shentong Mo, Xufang Luo, Dongsheng LiVisual Prompt LearningVision Foundation Model Adaptation

    Carnegie Mellon University · Microsoft Research

  27. MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation

    Feb 24, 2026Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh +1Multimodal Foundation ModelsPrompt Learning

    University of California, Santa Barbara · University of California, Los Angeles

  28. Test-Time Computing for Referring Multimodal Large Language Models

    Feb 23, 2026Mingrui Wu, Hao Chen, Jiayi Ji +5Test-Time Adaptation of VLMsMultimodal Large Language Models

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Zhongguancun Academy, Beijing, China.100094. · Tsinghua University. +1

  29. Prompt Tuning for CLIP on the Pretrained Manifold

    Feb 22, 2026Xi Yang, Yuanrong Xu, Weigang Zhang +3VLM AdaptationVisual Prompt Learning

    Guizhou University, Guiyang, China · Harbin Institute of Technology, China · The Chinese University of Hong Kong, Shenzhen, China

  30. Defect-aware Hybrid Prompt Optimization via Progressive Tuning for Zero-Shot Multi-type Anomaly Detection and Segmentation

    Dec 10, 2025Nadeem Nazer, Hongkuan Zhou, Lavdim Halilaj +2Industrial Anomaly DetectionMultimodal Anomaly Detection

    Corporate Research, Robert Bosch GmbH, Renningen, Germany · Otto-von-Guericke-University Magdeburg, Magdeburg, Germany · Institute for Artificial Intelligence, University of Stuttgart, Stuttgart, Germany +2