Prompt Learning

Follow this topic Ask about this topic

Prompt learning studies how to steer pretrained models by changing the instructions or prompt representations they receive, rather than retraining all their weights. Approaches range from hand-written hard prompts to trainable soft prompts, including methods such as CoOp and CoCoOp, as well as prompt pools, routing, and low-rank prompt tuning. These methods support efficient adaptation in settings such as few-shot CLIP classification, open-vocabulary recognition, out-of-distribution detection, and continual learning, where data or compute may be limited. A central challenge is balancing specialization to a task or domain against generalization to new classes and changing inputs, while keeping prompt selection reliable and adaptations interpretable.

Latest papers

78 papers9 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

Gram-Anchored Prompt Learning for Vision-Language Models via Second-Order Statistics
Apr 5, 2026Minglei Chen, Weilong Wang, Jiang Duan +1Prompt Learning for VLMs Prompt Learning
All authors are with Southwestern University of Finance and Economics, Chengdu, Sichuan, China.
ProTPS: Prototype-Guided Text Prompt Selection for Continual Learning
Apr 1, 2026Jie Mei, Li-Leng Peng, Keith Fuller +1Continual Learning for VLMs Class-Incremental Learning
University of Washington, Seattle, WA, USA · Alaska Pacific University, Anchorage, Alaska, USA
SkeletonContext: Skeleton-side Context Prompt Learning for Zero-Shot Skeleton-based Action Recognition
Mar 31, 2026Ning Wang, Tieyue Wu, Naeha Sharif +5Skeleton-Based Action Recognition Prompt Learning
Chang’an University · Xidian University · University of Western Australia +1
CrossHGL: A Text-Free Foundation Model for Cross-Domain Heterogeneous Graph Learning
Mar 29, 2026Xuanze Chen, Jiajun Zhou, Yadong Li +2Graph Representation Learning Graph Foundation Models
Xuanze Chen and Yadong Li are with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China, and also with the Binjiang Cyberspace Security Institute of ZJUT, Hangzhou, 310056, China · Jiajun Zhou, Shanqing Yu and Qi Xuan are with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China, with the Binjiang Cyberspace Security Institute of ZJUT, Hangzhou, 310056, China, and with the Soovar Technologies Co., Ltd., Hangzhou 310056, China
Exploring Multimodal Prompts For Unsupervised Continuous Anomaly Detection
Mar 23, 2026Mingle Zhou, Jiahui Liu, Jin Wan +2Continual Anomaly Detection Unsupervised Anomaly Detection
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China · Faculty of Data Science, City University of Macau, Macau, China
Boosting Quantitive and Spatial Awareness for Zero-Shot Object Counting
Mar 17, 2026Da Zhang, Bingyu Li, Feiyu Wang +2Object Counting Zero-Shot Learning
Northwestern Polytechnical University · Institute of Artificial Intelligence (TeleAI), China Telecom · University of Science and Technology of China +1
Distance-aware Soft Prompt Guidance for Multimodal Valence-Arousal Estimation
Mar 12, 2026Byeongjin Jung, Chanyeong Park, Sejoon LimValence-Arousal Modeling Multimodal Emotion Recognition
Kookmin University, Republic Of Korea
PromptDLA: A Domain-aware Prompt Document Layout Analysis Framework with Descriptive Knowledge as a Cue
Mar 10, 2026Zirui Zhang, Yaping Zhang, Lu Xiang +4Document Layout Analysis Domain Generalization
Columbia University · Institute of Automation, Chinese Academy of Sciences · University of the Chinese Academy of Sciences +1
Structure-aware Prompt Adaptation from Seen to Unseen for Open-Vocabulary Compositional Zero-Shot Learning
Mar 4, 2026Yihang Duan, Jiong Wang, Pengpeng Zeng +5Zero-Shot Learning Prompt Learning
University of Electronic Science and Technology of China · Ningbo University · Southwest Jiaotong University +1
Learning Domain-Aware Task Prompt Representations for Multi-Domain All-in-One Image Restoration
Mar 2, 2026Guanglu Dong, Chunlei Li, Chao Ren +4Image Restoration All-in-One Image Restoration
Sichuan University · MedAI Technology (Wuxi) Co. Ltd. · Technical University of Munich
DynaTokens: Controlling Token Dynamics for Continual Video-Language Understanding
Mar 2, 2026Toan Nguyen, Yang Liu, Celso De Melo +1Continual Learning Video QA
School of Computer Science and Engineering, University of New South Wales, Australia · DEVCOM Army Research Laboratory, USA
MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation
Feb 24, 2026Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh +1Multimodal Foundation Models Prompt Learning
University of California, Santa Barbara · University of California, Los Angeles
GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning
Feb 22, 2026Zehao Deng, An Liu, Yan WangZero-Shot Anomaly Detection Point Cloud Anomaly Detection
School of Computer Science and Technology, Soochow University · Institute for AI Industry Research (AIR), Tsinghua University
From Subtle to Significant: Prompt-Driven Self-Improving Optimization in Test-Time Graph OOD Detection
Feb 19, 2026Luzhi Wang, Xuanshuo Fu, He Zhang +3Graph Prompt Learning OOD Detection
College of Artificial Intelligence, Dalian Maritime University · Computer Vision Center, Universitat Aut`onoma de Barcelona · School of Computing Technologies, RMIT University +1
Panoramic Out-of-Distribution Segmentation
May 6, 2025Mengfei Duan, Yuheng Zhang, Yihong Cao +5OOD Detection Semantic Segmentation
The authors are with the School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China
Learning Personalized Prompts for Healthcare Guidance
Dec 20, 2024Ruize Shi, Hong Huang, Wei Zhou +3Prompt Learning LLM Personalization
Huazhong University of Science and Technology, Wuhan, China · 1Huazhong University of Science and Technology, Wuhan, China · 2Tongji Medical College +1
Detecting Conversational Mental Manipulation with Intent-Aware Prompting
Dec 11, 2024Jiayuan Ma, Hongbin Na, Zimu Wang +4Prompt Learning Mental Health
The University of Sydney · University of Technology Sydney · Xi’an Jiaotong-Liverpool University +2
Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation
Jul 23, 2024Xinghao Wu, Xuefeng Liu, Jianwei Niu +4Prompt Learning Federated Learning
School of Computer Science and Engineering, Beihang University, Beijing, China · Zhongguancun Laboratory, Beijing, China · Sichuan University, Sichuan, China +1

Apr 5, 2026J/K move · Enter open · S save

Gram-Anchored Prompt Learning for Vision-Language Models via Second-Order Statistics

Minglei Chen, Weilong Wang, Jiang Duan, Ye Deng

All authors are with Southwestern University of Finance and Economics, Chengdu, Sichuan, China.

Prompt Learning for VLMs Prompt Learning VLM Adaptation

Open paper AskPDF arXiv

Parameter-efficient prompt learning has become the de facto standard for adapting Vision-Language Models (VLMs) to downstream tasks. Existing approaches predominantly focus on aligning text prompts with first-order visual features (i.e., spatial feature maps). While effective for fine-grained semantic discrimination, we argue that relying solely on first-order information is insufficient for robust adaptation, as these spatially entangled features are highly susceptible to domain shifts and local noise. In this work, we propose Gram-Anchored Prompt Learning (GAPL) for Vision-Language Models via Second-Order Statistics, a framework that synergizes local semantic alignment with global structural consistency. Methodologically, we introduce an additional second-order statistical stream via Gram matrices that augments the standard first-order spatial interaction. By anchoring prompts to these second-order priors, our approach enables language representations to dynamically adapt to statistical distribution shifts across diverse domains. Extensive experiments indicate the effectiveness of the second-order features, and show compelling performances of GAPL on various benchmarks.