VLM Adaptation

Vision-language model adaptation studies how to specialize pretrained models such as CLIP and instruction-tuned multimodal large language models when target data, labels, or compute are limited. Common approaches include prompt tuning, LoRA and lightweight adapters, domain-specific fine-tuning, and test-time methods that use unlabeled examples or prediction memories; newer work also adapts models for video, medical imaging, remote sensing, and robotics. Evaluation often tests few-shot transfer, base-to-novel classes, cross-dataset generalization, robustness to distribution shift, and retention of the model’s original abilities. A central tension is that stronger specialization can cause forgetting, miscalibration, or reliance on language shortcuts, while conservative updates may preserve generality but limit task gains.