Prompt Tuning
Prompt tuning is a parameter-efficient fine-tuning technique for adapting large pre-trained models, such as vision-language models (VLMs) and large language models (LLMs), to specific downstream tasks by learning small sets of parameters (prompts) rather than retraining the entire model. Current research focuses on improving prompt design for various modalities (text, image, multimodal), enhancing calibration and robustness, and exploring applications across diverse fields including image segmentation, code repair, and recommendation systems. This approach offers significant advantages in terms of computational efficiency and reduced risk of overfitting, making it a valuable tool for adapting powerful foundation models to specialized tasks with limited data.
Papers
Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation
Chen Dun, Mirian Hipolito Garcia, Guoqing Zheng, Ahmed Hassan Awadallah, Anastasios Kyrillidis, Robert Sim
Inclusive Data Representation in Federated Learning: A Novel Approach Integrating Textual and Visual Prompt
Zihao Zhao, Zhenpeng Shi, Yang Liu, Wenbo Ding