cs.CVAug 30, 2025

Language-Aware Information Maximization for Transductive Few-Shot CLIP

Authors: Ghassen Baklouti, Maxime Zanella, Ismail Ben Ayed

Organizations: École de Technologie Supérieure (ÉTS), Montréal, Canada · Université Catholique de Louvain (UCLouvain), Louvain-La-Neuve, Belgium · Université de Mons (UMons), Mons, Belgium

Abstract

Transductive few-shot learning has triggered an abundant literature focusing on vision-only models, but is still at a nascent stage within the recent context of foundational vision-language models (VLMs). Only a few recent methods addressed the problem, pointing to the potential of tranduction in VLMs and to the need for VLM-tailored methods. Building on this momentum, we leverage information-theoretic concepts and recent progress in parameter-efficient fine-tuning (PEFT), developing a highly competitive transductive few-shot CLIP method. Specifically, we introduce a novel Language-aware Information MaximizatiOn (LIMO) loss integrating three complementary terms: (i) the mutual information between the vision inputs and the textual class descriptions; (ii) a Kullback-Leibler (KL) divergence penalizing deviation of the network's probabilistic outputs from the text-driven zero-shot predictions; and (iii) a standard cross-entropy loss based on the labeled shots. Furthermore, we challenge the commonly followed fine-tuning practices in the context of transductive few-shot learning, and explore PEFT strategies, completely overlooked in this context. Surprisingly, we observe substantial boosts in performances, which points to the potential of adapting a subset of the model's parameters in the transductive few-shot setting. We report comprehensive evaluations, which show that LIMO outperforms the very recent transductive few-shot CLIP methods by a large margin and yields significant gains over the best-performing inductive methods. Our code is publicly available at:https://github.com/ghassenbaklouti/LIMO

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Unlocking Few-Shot Capabilities in LVLMs via Prompt Conditioning and Head Selection

    Mar 25, 2026Adhemar de Senneville, Xavier Bou, Jérémy Anger +2Large Vision-Language ModelsZero-Shot Image Classification

  2. LE-GDA: Layer Ensemble Gaussian Discriminant Analysis for Training-Free Few-Shot CLIP Adaptation

    Oct 26, 2026Pavodi Maniamfu, Vogel Kiketa, Zezina Kuzulu Kuetu +4VLM AdaptationFew-Shot Image Classification

  3. Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning

    May 28, 2026Shuai Yi, Yixiong Zou, Yuhua Li +1Source-Free Cross-Domain Few-Shot LearningSource-Free Domain Adaptation