Few-Shot VLM Adaptation

VLM: Vision-Language Model

Latest papers

9 papers0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

On the Few-Shot Adaptation of Vision-Language Models: A Few Gradient Signs Are All You Need
Nov 5, 2026Nairouz Mrabah, Ismail Ben Ayed, Mahdi S. Hosseini +1Few-Shot VLM Adaptation VLM Adaptation
Department of Systems Engineering, ETS Montreal, Canada · Department of Computer Science and Software Engineering, Concordia, Canada
SD-LoRA: Training-Time Structural Distillation into LoRA for Few-Shot Vision-Language Adaptation
Nov 4, 2026Mohammed Rahman Sherif Khan Mohammad, Ardhendu Behera, Sandip Pradhan +2Few-Shot VLM Adaptation VLM Adaptation
Edge Hill University, UK
Learning to Adapt and Calibrate: Score Distribution Alignment for Few-Shot Uncertainty Prediction in Medical VLMs
Sep 9, 2026Xuan Cuong Ngo, Ngan LeVLM Adaptation Few-Shot VLM Adaptation
University of Arkansas Fayetteville, Arkansas, USA
The Blending Ratio Is Not Where the Performance Is: Diagnosing Prototype Blending for Few-Shot Adaptation of Vision-Language Models
Aug 23, 2026Liangzhi Li, Bowen Wang, Yiming Qian +3Few-Shot VLM Adaptation VLM Adaptation
Department of Computer Science, Qufu Normal University, China, and with Climind · Institute of Scientific and Industrial Research (SANKEN), The University of Osaka, Japan · Institute of High Performance Computing (IHPC), A*STAR, Singapore +3
PRiSM: Prototype Regularization for Few-Shot VLMs
Jul 20, 2026Ghassen Baklouti, Omprakash Chakraborty, Jose Dolz +1VLM Adaptation Prototype Learning
ÉTS Montreal
BioMedVR: Confusion-Aware Mixture-of-Prompt Experts for Biomedical Visual Reprogramming
Jun 23, 2026Jiaxiang Liu, Tianxiang Hu, Juwei Guan +5VLM Adaptation Few-Shot VLM Adaptation
Guangdong Institute of Intelligence Science and Technology, Zhuhai, China · Zhejiang University, Hangzhou, China · Southeast University, Nanjing, China +3
Direct Product Flow Matching: Decoupling Radial and Angular Dynamics for Few-Shot Adaptation
May 6, 2026Hongxu Chen, Yanghao Wang, Bowei Zhu +6Cross-Modal Learning Flow Matching
HKUST · USTC · Huawei Research
Hold-One-Shot-Out (HOSO) for Validation-Free Few-Shot CLIP Adapters
Mar 4, 2026Chris Vorster, Mayug Maniparambil, Noel E. O'Connor +2Few-Shot VLM Adaptation VLM Adaptation
ML-Labs, Dublin City University, Dublin, Ireland
Path-Decoupled Hyperbolic Flow Matching for Few-Shot Adaptation
Feb 24, 2026Lin Li, Ziqi Jiang, Gefan Ye +5Few-Shot VLM Adaptation VLM Adaptation
The Hong Kong University of Science and Technology (HKUST) · Zhejiang University

Nov 5, 2026J/K move · Enter open · S save

On the Few-Shot Adaptation of Vision-Language Models: A Few Gradient Signs Are All You Need

Nairouz Mrabah, Ismail Ben Ayed, Mahdi S. Hosseini, Eric Granger

Department of Systems Engineering, ETS Montreal, Canada · Department of Computer Science and Software Engineering, Concordia, Canada

Few-Shot VLM Adaptation VLM Adaptation Deep Learning Optimization Stochastic Gradient Descent

Open paper AskPDF arXiv

The limited supervision available for the few-shot adaptation of vision-language models makes gradient estimation particularly noisy. We find that the gradient magnitudes are severely miscalibrated, while the signs of a small top-magnitude subset remain informative. This motivates SignSparseSGD and DynSignSparseSGD, two state-free optimizers that use gradient magnitudes only to select where to update and fixed-magnitude signs to decide how to update. The static variant fixes the support after the first minibatch. The dynamic variant refreshes it at every step. Across 11 datasets, three CLIP backbones, and five shot levels, the proposed methods are competitive with or outperform parameter-efficient and optimization-based baselines on average, while exhibiting markedly lower sensitivity to long training schedules. They introduce no task-specific modules, maintain no first- or second-moment optimizer states, and use one shared configuration across datasets, backbones, and shot levels. Our code is included in the supplementary materials and will be released upon publication.