Prompt Tuning

Follow this topic Ask about this topic

Prompt tuning studies how small learned vectors, inserted into a model’s inputs or internal layers, can adapt a largely frozen pretrained model to new tasks. Common approaches include soft prompts and prefixes, structured or task-specific prompt banks, and routing or test-time methods that select or adjust prompts for each input or domain. These techniques are used with language and vision-language models as well as audio, graphs, and time series, often aiming to reduce training and memory costs while retaining pretrained capabilities. A central challenge is balancing effective adaptation with robustness and generalization: prompts can overfit limited or noisy data, interfere with existing behavior, or behave unpredictably across shifts, and their internal effects are not always well understood.

Latest papers

76 papers10 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Acoustic Prompting via Stage-wise Modulation for Few-Shot Learning in Audio Language Models
Jun 14, 2026Hyebin Cho, Jaehyuk Jang, Changick Kim +1Few-Shot Audio Classification Audio Representation Learning
Korea Advanced Institute of Science and Technology, South Korea
Distilling Safe LLM Systems via Soft Prompts for On Device Settings
Jun 8, 2026Motasem Alfarra, Cristina Pinneri, Dana Kianfar +2On-Device Language Model Inference LLM Safety Alignment
Qualcomm AI Research
GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding
Jun 5, 2026Minseong Kim, Jinyeong Park, Sungho Park +13D Asset Generation Parametric CAD Modeling
Convergence Research Center for Insect Vectors Incheon, South Korea · Department of Computer Science and Engineering Incheon National University Incheon, 22012, South Korea · Department of Computer Science and Engineering Center for Brain-Machine Interface Incheon National University Incheon, South Korea
Beyond Retrieval: Learning Compact User Representations for Scalable LLM Personalization
Jun 3, 2026Heng Cao, Fan Zhang, Jian Yao +8LLM Personalization Prompt Tuning
*Microsoft · †The Hong Kong Polytechnic University · ⋄Shanghai International Studies University +1
Message Tuning Outshines Graph Prompt Tuning: A Prismatic Space Perspective
Jun 2, 2026Yancheng Chen, Dun Ma, Shuai Zhang +6Prompt Tuning Graph Foundation Models
Academy of Mathematics and Systems Science, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Institute of Information Engineering, Chinese Academy of Sciences +4
TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models
Jun 2, 2026Wenbo Zhang, Jianxiong Li, Shuai Yang +4Test-Time Adaptation Test-Time Adaptation of VLMs
ByteDance Seed · The University of Adelaide · Tsinghua University +3
Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models
May 30, 2026S M Tahmid Siddiqui, Akib Jawad Ononto, Anoop Singhal +1LLM Hallucination Mitigation Question Answering
The University of Texas at Dallas, Richardson, TX 75080, USA · National Institute of Standards and Technology, Gaithersburg, USA
EntroAD: Structural Entropy-Guided Prompt Adaptation for Zero-Shot Anomaly Detection
May 27, 2026Xinyu Zhao, Qingyun Sun, Jiayi Luo +1VLM Adaptation Medical Image Anomaly Detection
Beihang University Beijing, China
Let Relations Speak: An End-to-End LLM-GNN Soft Prompt Framework for Fraud Detection
May 27, 2026Zhixing Zuo, Huilin He, Jiasheng Wu +1Graph Neural Networks Graph Representation Learning
School of Computer Science and Technology, Tongji University
PromptEmbedder: Efficient and Transferable Text Embedding via Dual-LLM Soft Prompting
May 27, 2026Yu-Che Tsai, Kuan-Yu Chen, Yuan-Hao Chen +4Text Embeddings Prompt Tuning
Department of Computer Science and Information Engineering, National Taiwan University · National Taiwan University AI Center of Research Excellence Taipei, Taiwan
CMAP: Cross-Modal Adaptive Prompting for Multi-Domain Task-Incremental Learning
May 25, 2026Sriram MandalikaCross-Modal Learning VLM Adaptation
Hasso Plattner Institute Potsdam, Germany
Turning Adaptation into Assets: Cross-Domain Bridging for Online Vision-Language Navigation
May 22, 2026Zixuan Hu, Xuantuo Huang, Yancheng Li +3VLM Adaptation Continual Test-Time Adaptation
School of Computer Science, Peking University, Beijing, China · School of Electronics, Peking University, Beijing, China · Peng Cheng Laboratory, Shenzhen, China
MixFake: Benchmarking and Enhancing Audio Deepfake Detection in Diverse Real-world Mixed Audio
May 22, 2026Qingcao Li, Yipeng Lin, Weichen Lian +3Audio Deepfake Detection Prompt Tuning
School of Cyber Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China
PromptRad: Knowledge-Enhanced Multi-Label Prompt-Tuning for Low-Resource Radiology Report Labeling
May 19, 2026Ying-Jia Lin, Tzu-Chin Lo, Ping-Chien Li +3Multi-Label Classification Healthcare
Department of Artificial Intelligence and AI Research Center, Chang Gung University · Department of Radiology, Sijhih Cathay General Hospital · Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital +2
CLAP: Contrastive Latent-space Prompt Optimization for End-to-end Autonomous Driving
May 17, 2026Ruiyang Zhu, Yuehan He, Boyuan Zheng +4Long-Tail Autonomous Driving VLMs for Autonomous Driving
University of Michigan · University of Arizona
SteerSeg: Attention Steering for Reasoning Video Segmentation
May 14, 2026Ali Cheraghian, Hamidreza Dastmalchi, Abdelwahed Khamis +3Video Object Segmentation Referring Video Object Segmentation
1Macquarie University · 2York University · 3CSIRO Data61 +1
GFMate: Empowering Graph Foundation Models with Test-time Prompt Tuning
May 14, 2026Yan Jiang, Ruihong Qiu, Zi HuangDomain Adaptation Graph Domain Adaptation
School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, Queensland, Australia
PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts
May 13, 2026Anjir Ahmed Chowdhury, Syed Zawad, Xiaolong Ma +2LLM Fine-Tuning Multi-Task Learning
Department of Computer Science University of Houston · IBM Research · Argonne National Laboratory
Decoupled and Divergence-Conditioned Prompt for Multi-domain Dynamic Graph Foundation Models
May 13, 2026Haonan Yuan, Qingyun Sun, Junhua Shi +3Domain-Adaptive Pretraining Prompt Tuning
School of Computer Science and Engineering, Beihang University, Beijing 100191, China · Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin 541004, China · Department of Computer Science, University of Illinois at Chicago, Chicago 60607, USA
Learning Perturbations to Extrapolate Your LLM
May 13, 2026Zetai Cen, Chenfei Gu, Jin Zhu +3OOD Generalization Prompt Tuning
School of Mathematics, University of Bristol · School of Statistics and Data Science, Shanghai University of Finance and Economics · School of Mathematics, University of Birmingham +1
How You Begin is How You Reason: Driving Exploration in RLVR via Prefix-Tuned Priors
May 9, 2026Yifan Xu, Junren Chen, Yifan ChenRL for Language Model Reasoning Prompt Tuning
Hong Kong Baptist University · University of Maryland, College Park
Efficient Prompt Learning for Traffic Forecasting
May 8, 2026Qianru Zhang, Xinyi Gao, Alexander Zhou +3OOD Generalization Prompt Tuning
School of Computing and Data Science The University of Hong Kong · School of Electrical Engineering and Computer Science The University of Queensland · Department of Computing The Hong Kong Polytechnic University
RFPrompt: Prompt-Based Expert Adaptation of the Large Wireless Model for Modulation Classification
May 5, 2026Md Raihan Uddin, Tolunay Seyfi, Fatemeh AfghahWireless Foundation Models Automatic Modulation Classification
Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA
When Prompts Interact: Assessing Prompt Arithmetic for Deconfounding under Distribution Shift
May 4, 2026Zhecheng Sheng, Yongsen Tan, Xiruo Ding +2Distribution Shift Robustness Prompt Tuning
University of Minnesota · University of Washington · 3Stanford University
Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Models
May 1, 2026Jiayu Li, Jiaxin Qi, Sheng Zhou +2VLM Robustness VLM Adaptation
Enhancing Continual Learning of Vision-Language Models via Dynamic Prefix Weighting
Apr 20, 2026Hyeonseo Jang, Hyuk Kwon, Kibok LeeClass-Incremental Learning Vision-Language Models
Yonsei University
RePrompT: Recurrent Prompt Tuning for Integrating Structured EHR Encoders with Large Language Models
Apr 20, 2026Arya Hadizadeh Moghaddam, Drew Ross, Mohsen Nayebi Kerdabadi +2Representation Learning Prompt Tuning
University of Kansas, USA
SCHK-HTC: Sibling Contrastive Learning with Hierarchical Knowledge-Aware Prompt Tuning for Hierarchical Text Classification
Apr 17, 2026Ke Xiong, Qian Wu, Wangjie Gan +2Hierarchical Classification Hierarchical Representation Learning
School of Software Technology, Zhejiang University, China · NetEase Yidun AI Lab, Hangzhou, China
Your Pre-trained Diffusion Model Secretly Knows Restoration
Apr 6, 2026Sudarshan Rajagopalan, Vishal M. PatelDiffusion Models for Image Restoration All-in-One Image Restoration
Johns Hopkins University, Baltimore MD 21218, USA
Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection
Apr 6, 2026Weihao Cao, Runqi Wang, Xiaoyue Duan +3Open-Vocabulary Object Detection Vision Foundation Model Adaptation
State Key Laboratory of Advanced Rail Autonomous Operation, Beijing Jiaotong University · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University · School of Computer Science & Technology, Beijing Jiaotong University +1

Jun 14, 2026J/K move · Enter open · S save

Acoustic Prompting via Stage-wise Modulation for Few-Shot Learning in Audio Language Models

Hyebin Cho, Jaehyuk Jang, Changick Kim, Joon Son Chung

Korea Advanced Institute of Science and Technology, South Korea

Few-Shot Audio Classification Audio Representation Learning Few-Shot Learning Audio-Language Models Prompt Tuning Audio Classification

Open paper AskPDF arXiv

Audio-Language Models (ALMs) have shown remarkable success in zero-shot audio classification by aligning audio waveforms with text. Recent efforts to improve downstream performance focus on learning optimal text prompts. However, previous approaches focus on the text encoder, leaving the potential of learnable prompts within the audio encoder unexplored. In this paper, we propose a novel framework that introduces trainable prompts into the audio encoder to capture task-specific acoustic features. We demonstrate that integrating audio-side prompt learning with existing text-side approaches enhances few-shot adaptation. Through extensive experiments across 11 datasets show that integrating our method as a plug-and-play module alongside existing text prompt tuning generally leads to performance improvements. These findings suggest that explicitly modulating the audio representation space effectively complements text-only prompting approaches. The code is available at https://github.com/hyebin-c/aspl.