Few-Shot Domain Adaptation

Follow this topic Ask about this topic

Few-shot domain adaptation studies how to transfer a model to settings where the target data differ from its training data and labels are scarce, such as new hospitals, sensors, users, or environments. Common approaches fine-tune all or part of a pretrained model, use lightweight adapters or prompts, align features with prototypes, or augment sparse examples with pseudo-labels and synthetic data. Researchers evaluate these methods on tasks including image classification, object detection, segmentation, speech, and sensor recognition, increasingly using foundation models and source-free settings where original training data are unavailable. A central tension is gaining target-domain accuracy without overfitting to a few examples or erasing useful pretrained capabilities; results also depend strongly on the type and severity of the domain shift.

Latest papers

53 papers9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

May 28, 2026

GiPL: Generative augmented iterative Pseudo-Labeling for Cross-Domain Few-Shot Object Detection

Vision-language foundation models have shown promising zero-shot generalization for Cross-Domain Few-Shot Object Detection (CD-FSOD). However, they face two critical challenges in fine-tuning: insufficient support set utilization due to sparse single-instance annotations, and severe overfitting under extremely limited target-domain samples. To address these issues, this paper proposes GiPL, an efficient two-branch training framework. In the first branch, we design an iterative pseudo-label self-training paradigm, which performs zero-shot inference on the support set to generate reliable pseudo-annotations, fuses them with ground-truth labels, and iteratively optimizes the model to fully exploit support set data. In the second branch, we introduce generative data augmentation pipeline using large vision-language models, which synthesizes domain-aligned, multi-object annotated images to enrich training samples and suppress overfitting. Extensive experiments on three challenging CD-FSOD datasets (RUOD, CARPK, CarDD) under 1/5/10-shot settings demonstrate that GiPL consistently outperforms state-of-the-art methods with significant performance gains. Code is available at CDiscover.

Jiacong Liu, Shu Luo, Yikai Qin +3

Huazhong University of Science and Technology

Synthetic Data Augmentation Few-Shot Domain Adaptation Vision Foundation Model Adaptation Few-Shot Object Detection Few-Shot Learning Pseudo-Labeling+1

May 26, 2026

FSA-GRPO: Teaching Auditory LLMs to Use Few-shot Demonstrations

Few-shot prompting provides an effective way to adapt auditory large language models to low-resource tasks such as children's speech recognition. However, most auditory large language models are not explicitly trained to perform inference in this demonstration-conditioned format, limiting the extent to which they can benefit from few-shot prompting. To address this limitation, we introduce Few-Shot Aware GRPO (FSA-GRPO), an RL-based post-training recipe that uses a specially designed reward to encourage the model to leverage few-shot demonstrations, thereby strengthening its few-shot adaptation ability. Notably, training with only high-resource adult ASR data improves the model's general few-shot adaptation ability, yielding gains not only in children's speech recognition but also in speech translation and audio understanding. We further study data selection and auxiliary reward weighting to identify an effective training recipe. Our experiments show that when in-domain data are unavailable or cannot be used for training, FSA-GRPO is more effective than direct tuning on related out-of-domain data.

Haolong Zheng, Siyin Wang, Xulin Fan +2

University of Illinois Urbana Champaign · Tsinghua University

RL for Language Models Few-Shot Domain Adaptation Few-Shot Learning Audio-Language Models In-Context Learning Low-Resource Speech Recognition

May 25, 2026

Addressing Exacerbated Attention Sink for Source-Free Cross-Domain Few-Shot Learning

Vision-language models (VLMs) like CLIP have shown impressive generalization capabilities, yet their potential for Cross-Domain Few-Shot Learning (CDFSL) remains underexplored, where the model needs to transfer source-domain information to target domains with scarce training data. While the attention sink phenomenon has been observed in VLMs for certain tasks, its role in CDFSL scenarios has not been studied. In this paper, we uncover a critical issue overlooked by prior works: standard target-domain few-shot fine-tuning in CDFSL significantly exacerbates the attention sink problem, leading to poor discriminability across classes. To understand this phenomenon, through extensive experiments, we interpret it as the model's shortcut learning for domain adaptation: to overcome the huge domain gap between the source and target domains, the model shows a high tendency to push tokens that are initially closer to target-domain classes (i.e., simple tokens) to be even closer to these classes, exacerbating the attention sink and wasting the capability of learning other discriminative but initially further tokens (i.e., hard tokens). To address this, we propose a novel approach to dynamically re-weight tokens according to their relevance with target-domain classes during the target-domain finetuning, which explicitly suppresses the model's reliance on these simple tokens and enhances the learning of hard tokens, reducing sink tokens and enhancing discriminability. Extensive experiments on four benchmark datasets validate the rationale of our method, demonstrating new state-of-the-art performance. Our codes are available at https://github.com/shuaiyi308/TIR.

Shuai Yi, Yixiong Zou, Yuhua Li +1

Huazhong University of Science and Technology

Source-Free Cross-Domain Few-Shot Learning Source-Free Domain Adaptation Few-Shot Domain Adaptation VLM Adaptation Few-Shot Learning Attention Sinks

May 12, 2026

Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning

Cross-Domain Few-Shot Learning (CDFSL) aims to adapt large-scale pretrained models to specialized target domains with limited samples, yet the few-shot fine-tuning of vision-language models like CLIP remains underexplored. By establishing multiple fine-tuning baselines of CLIP for CDFSL, we find adapter-based methods (e.g., LoRA) consistently outperform prompt-based ones (e.g., MaPLe), contrary to in-domain scenarios. To make those effective in-domain methods competitive again in CDFSL, we analyze this phenomenon and discover LoRA's superiority stems from rectifying the collapsed attention of visual CLS token, enhancing modality alignment and class separation by focusing on text-related visual regions. Further, we find textual EOS token exhibit much better attention to visual samples, and CLIP's standard contrastive loss weakly constrains modality alignment. Based on these insights, we propose Semantic Probe, a plug-and-play attention rectification framework for both adapter- and prompt-based methods. Extensive experiments on four CDFSL benchmarks validate our rationale, achieving state-of-the-art performance and benefiting both fine-tuning paradigms. Codes will be released.

Yaze Zhao, Yicong Liu, Yixiong Zou +2

Source-Free Cross-Domain Few-Shot Learning Source-Free Domain Adaptation Few-Shot Domain Adaptation VLM Adaptation Few-Shot Learning Vision-Language Alignment+1

Apr 13, 2026

The Second Challenge on Cross-Domain Few-Shot Object Detection at NTIRE 2026: Methods and Results

Cross-domain few-shot object detection (CD-FSOD) remains a challenging problem for existing object detectors and few-shot learning approaches, particularly when generalizing across distinct domains. As part of NTIRE 2026, we hosted the second CD-FSOD Challenge to systematically evaluate and promote progress in detecting objects in unseen target domains under limited annotation conditions. The challenge received strong community interest, with 128 registered participants and a total of 696 submissions. Among them, 31 teams actively participated, and 19 teams submitted valid final results. Participants explored a wide range of strategies, introducing innovative methods that push the performance frontier under both open-source and closed-source tracks. This report presents a detailed overview of the NTIRE 2026 CD-FSOD Challenge, including a summary of the submitted approaches and an analysis of the final results across all participating teams. Challenge Codes: https://github.com/ohMargin/NTIRE2026_CDFSOD.

Xingyu Qiu, Yuqian Fu, Jiawei Geng +71

Few-Shot Object Detection Object Detection Few-Shot Domain Adaptation

Apr 10, 2026

Adaptor: Advancing Assistive Teleoperation with Few-Shot Learning and Cross-Operator Generalization

Assistive teleoperation enhances efficiency via shared control, yet inter-operator variability, stemming from diverse habits and expertise, induces highly heterogeneous trajectory distributions that undermine intent recognition stability. We present Adaptor, a few-shot framework for robust cross-operator intent recognition. The Adaptor bridges the domain gap through two stages: (i) preprocessing, which models intent uncertainty by synthesizing trajectory perturbations via noise injection and performs geometry-aware keyframe extraction; and (ii) policy learning, which encodes the processed trajectories with an Intention Expert and fuses them with the pre-trained vision-language model context to condition an Action Expert for action generation. Experiments on real-world and simulated benchmarks demonstrate that Adaptor achieves state-of-the-art performance, improving success rates and efficiency over baselines. Moreover, the method exhibits low variance across operators with varying expertise, demonstrating robust cross-operator generalization.

Yu Liu, Yihang Yin, Tianlv Huang +8

School of Artificial Intelligence, Jilin University. · IO-AI TECH.

Robot Teleoperation Robot Learning Assistive Robotics Few-Shot Domain Adaptation

Mar 29, 2026

CrossHGL: A Text-Free Foundation Model for Cross-Domain Heterogeneous Graph Learning

Heterogeneous graph representation learning (HGRL) is essential for modeling complex systems with diverse node and edge types. However, most existing methods are limited to closed-world settings with shared schemas and feature spaces, hindering cross-domain generalization. While recent graph foundation models improve transferability, they often target homogeneous graphs, rely on domain-specific schemas, or require rich textual attributes. Consequently, text-free and few-shot cross-domain HGRL remains underexplored. To address this, we propose CrossHGL, a foundation framework that preserves and transfers multi-relational structural semantics without external textual supervision. Specifically, a semantic-preserving transformation strategy homogenizes heterogeneous graphs while encoding interaction semantics into edge features. Based on this, a prompt-aware multi-domain pre-training framework with a Tri-Prompt mechanism captures transferable knowledge across feature, edge, and structure perspectives via self-supervised contrastive learning. For target-domain adaptation, we develop a parameter-efficient fine-tuning strategy that freezes the pre-trained backbone and performs few-shot classification via prompt composition and prototypical learning. Experiments on node-level and graph-level tasks show that CrossHGL consistently outperforms state-of-the-art baselines, yielding average relative improvements of 25.1% and 7.6% in Micro-F1 for node and graph classification, respectively, while remaining competitive in challenging feature-degenerated settings.

Xuanze Chen, Jiajun Zhou, Yadong Li +2

Xuanze Chen and Yadong Li are with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China, and also with the Binjiang Cyberspace Security Institute of ZJUT, Hangzhou, 310056, China · Jiajun Zhou, Shanqing Yu and Qi Xuan are with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China, with the Binjiang Cyberspace Security Institute of ZJUT, Hangzhou, 310056, China, and with the Soovar Technologies Co., Ltd., Hangzhou 310056, China

Graph Representation Learning Graph Foundation Models Few-Shot Domain Adaptation Graph SSL Prompt Learning

Mar 24, 2026

Few-Shot Generative Model Adaption via Identity Injection and Preservation

Training generative models with limited data presents severe challenges of mode collapse. A common approach is to adapt a large pretrained generative model upon a target domain with very few samples (fewer than 10), known as few-shot generative model adaptation. However, existing methods often suffer from forgetting source domain identity knowledge during adaptation, which degrades the quality of generated images in the target domain. To address this, we propose Identity Injection and Preservation (I

^2

P), which leverages identity injection and consistency alignment to preserve the source identity knowledge. Specifically, we first introduce an identity injection module that integrates source domain identity knowledge into the target domain's latent space, ensuring the generated images retain key identity knowledge of the source domain. Second, we design an identity substitution module, which includes a style-content decoupler and a reconstruction modulator, to further enhance source domain identity preservation. We enforce identity consistency constraints by aligning features from identity substitution, thereby preserving identity knowledge. Both quantitative and qualitative experiments show that our method achieves substantial improvements over state-of-the-art methods on multiple public datasets and 5 metrics.

Yeqi He, Liang Li, Jiehua Zhang +4

School of Cyberspace Security, Hangzhou Dianzi University, Hangzhou, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · School of Software Engineering, Xi’an Jiaotong University, Xi’an, China +3

Few-Shot Domain Adaptation Few-Shot Learning Identity-Preserving Image Generation

Mar 19, 2026

Remedying Target-Domain Astigmatism for Cross-Domain Few-Shot Object Detection

Cross-domain few-shot object detection (CD-FSOD) aims to adapt pretrained detectors from a source domain to target domains with limited annotations, suffering from severe domain shifts and data scarcity problems. In this work, we find a previously overlooked phenomenon: models exhibit dispersed and unfocused attention in target domains, leading to imprecise localization and redundant predictions, just like a human cannot focus on visual objects. Therefore, we call it the target-domain Astigmatism problem. Analysis on attention distances across transformer layers reveals that regular fine-tuning inherently shows a trend to remedy this problem, but results are still far from satisfactory, which we aim to enhance in this paper. Biologically inspired by the human fovea-style visual system, we enhance the fine-tuning's inherent trend through a center-periphery attention refinement framework, which contains (1) a Positive Pattern Refinement module to reshape attention toward semantic objects using class-specific prototypes, simulating the visual center region; (2) a Negative Context Modulation module to enhance boundary discrimination by modeling background context, simulating the visual periphery region; and (3) a Textual Semantic Alignment module to strengthen center-periphery distinction through cross-modal cues. Our bio-inspired approach transforms astigmatic attention into focused patterns, substantially improving adaptation to target domains. Experiments on six challenging CD-FSOD benchmarks consistently demonstrate improved detection accuracy and establish new state-of-the-art results.

Yongwei Jiang, Yixiong Zou, Yuhua Li +1

School of Computer Science and Technology, Huazhong University of Science and Technology

Few-Shot Object Detection Object Detection Few-Shot Domain Adaptation Visual Attention

Mar 18, 2026

Interpretable Cross-Domain Few-Shot Learning with Rectified Target-Domain Local Alignment

Cross-Domain Few-Shot Learning (CDFSL) adapts models trained with large-scale general data (source domain) to downstream target domains with only scarce training data, where the research on vision-language models (e.g., CLIP) is still in the early stages. Typical downstream domains, such as medical diagnosis, require fine-grained visual cues for interpretable recognition, but we find that current fine-tuned CLIP models can hardly focus on these cues, albeit they can roughly focus on important regions in source domains. Although current works have demonstrated CLIP's shortcomings in capturing local subtle patterns, in this paper, we find that the domain gap and scarce training data further exacerbate such shortcomings, much more than that of holistic patterns, which we call the local misalignment problem in CLIP-based CDFSL. To address this problem, due to the lack of supervision in aligning local visual features and text semantics, we turn to self-supervision information. Inspired by the translation task, we propose the CC-CDFSL method with cycle consistency, which translates local visual features into text features and then translates them back into visual features (and vice versa), and constrains the original features close to the translated back features. To reduce the noise imported by richer information in the visual modality, we further propose a Semantic Anchor mechanism, which first augments visual features to provide a larger corpus for the text-to-image mapping, and then shrinks the image features to filter out irrelevant image-to-text mapping. Extensive experiments on various benchmarks, backbones, and fine-tuning methods show we can (1) effectively improve the local vision-language alignment, (2) enhance the interpretability of learned patterns and model decisions by visualizing patches, and (3) achieve state-of-the-art performance.

Yaze Zhao, Yixiong Zou, Yuhua Li +1

School of Computer Science and Technology, Huazhong University of Science and Technology

Cycle Consistency Source-Free Cross-Domain Few-Shot Learning Few-Shot Learning Few-Shot Domain Adaptation VLM Adaptation Vision-Language Alignment+1

Mar 17, 2026

Sample-Efficient Adaptation of Drug-Response Models to Patient Tumors under Strong Biological Domain Shift

Predicting drug response in patients from preclinical data remains a major challenge in precision oncology due to the substantial biological gap between in vitro cell lines and patient tumors. Rather than aiming to improve absolute in vitro prediction accuracy, this work examines whether explicitly separating representation learning from task supervision enables more sample-efficient adaptation of drug-response models to patient data under strong biological domain shift. We propose a staged transfer-learning framework in which cellular and drug representations are first learned independently from large collections of unlabeled pharmacogenomic data using autoencoder-based representation learning. These representations are then aligned with drug-response labels on cell-line data and subsequently adapted to patient tumors using few-shot supervision. Through a systematic evaluation spanning in-domain, cross-dataset, and patient-level settings, we show that unsupervised pretraining provides limited benefit when source and target domains overlap substantially, but yields clear gains when adapting to patient tumors with very limited labeled data. In particular, the proposed framework achieves faster performance improvements during few-shot patient-level adaptation while maintaining comparable accuracy to single-phase baselines on standard cell-line benchmarks. Overall, these results demonstrate that learning structured and transferable representations from unlabeled molecular profiles can substantially reduce the amount of clinical supervision required for effective drug-response prediction, offering a practical pathway toward data-efficient preclinical-to-clinical translation.

Camille Jimenez Cortes, Philippe Lalanda, German Vega

Université Grenoble Alpes (UGA), France

Cancer Drug Response Prediction Cross-Domain Transfer Learning Few-Shot Domain Adaptation Clinical Prediction Molecular Representation Learning

Mar 13, 2026

Modality-free Graph In-context Alignment

In-context learning (ICL) converts static encoders into task-conditioned reasoners, enabling adaptation to new data from just a few examples without updating pretrained parameters. This capability is essential for graph foundation models (GFMs) to approach LLM-level generality. Yet current GFMs struggle with cross-domain alignment, typically relying on modality-specific encoders that fail when graphs are pre-vectorized or raw data is inaccessible. In this paper, we introduce Modality-Free Graph In-context Alignment (MF-GIA), a framework that makes a pretrained graph encoder promptable for few-shot prediction across heterogeneous domains without modality assumptions. MF-GIA captures domain characteristics through gradient fingerprints, which parameterize lightweight transformations that align pre-encoded features and indexed labels into unified semantic spaces. During pretraining, a dual prompt-aware attention mechanism with episodic objective learns to match queries against aligned support examples to establish prompt-based reasoning capabilities. At inference, MF-GIA performs parameter-update-free adaptation using only a few-shot support set to trigger cross-domain alignment and enable immediate prediction on unseen domains. Experiments demonstrate that MF-GIA achieves superior few-shot performance across diverse graph domains and strong generalization to unseen domains.

Wei Zhuo, Siqiang Luo

Nanyang Technological University

Graph Foundation Models In-Context Learning Graph Neural Networks Few-Shot Domain Adaptation

Mar 12, 2026

ReHARK: Refined Hybrid Adaptive RBF Kernels for Robust One-Shot Vision-Language Adaptation

The adaptation of large-scale Vision-Language Models (VLMs) like CLIP to downstream tasks with extremely limited data -- specifically in the one-shot regime -- is often hindered by a significant "Stability-Plasticity" dilemma. While efficient caching mechanisms have been introduced by training-free methods such as Tip-Adapter, these approaches often function as local Nadaraya-Watson estimators. Such estimators are characterized by inherent boundary bias and a lack of global structural regularization. In this paper, ReHARK (Refined Hybrid Adaptive RBF Kernels) is proposed as a synergistic training-free framework that reinterprets few-shot adaptation through global proximal regularization in a Reproducing Kernel Hilbert Space (RKHS). A multistage refinement pipeline is introduced, consisting of: (1) Hybrid Prior Construction, where zero-shot textual knowledge from CLIP and GPT-3 is fused with visual class prototypes to form a robust semantic-visual anchor; (2) Support Set Augmentation (Bridging), where intermediate samples are generated to smooth the transition between visual and textual modalities; (3) Adaptive Distribution Rectification, where test feature statistics are aligned with the augmented support set to mitigate domain shifts; and (4) Multi-Scale RBF Kernels, where an ensemble of kernels is employed to capture complex feature geometries across diverse scales. Superior stability and accuracy are demonstrated through extensive experiments on 11 diverse benchmarks. A new state-of-the-art for one-shot adaptation is established by ReHARK, which achieves an average accuracy of 65.83%, significantly outperforming existing baselines. Code is available at https://github.com/Jahid12012021/ReHARK.

Md Jahidul Islam

Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh

VLM Adaptation Test-Time Adaptation of VLMs Kernel Methods Few-Shot Domain Adaptation

Mar 11, 2026

Few-Shot Adaptation to Non-Stationary Environments via Latent Trend Embedding for Robotics

Robotic systems operating in real-world environments often suffer from concept shift, where the input-output relationship changes due to latent environmental factors that are not directly observable. Conventional adaptation methods update model parameters, which may cause catastrophic forgetting and incur high computational cost. This paper proposes a latent Trend ID-based framework for few-shot adaptation in non-stationary environments. Instead of modifying model weights, a low-dimensional environmental state, referred to as the Trend ID, is estimated via backpropagation while the model parameters remain fixed. To prevent overfitting caused by per-sample latent variables, we introduce temporal regularization and a state transition model that enforces smooth evolution of the latent space. Experiments on a quantitative food grasping task demonstrate that the learned Trend IDs are distributed across distinct regions of the latent space with temporally consistent trajectories, and that few-shot adaptation to unseen environments is achieved without modifying model parameters. The proposed framework provides a scalable and interpretable solution for robotics applications operating across diverse and evolving environments.

Yasuyuki Fujii, Emika Kameda, Hiroki Fukada +3

College of Information Science and Engineering, Ritsumeikan University, Osaka, Japan · College of Information Science and Engineering, Ritsumeikan University, Osaka, Japan. · Production and Technology Department, NIPPN CORPORATION, Tokyo, Japan +5

Robot Policy Adaptation Few-Shot Domain Adaptation State Representation Learning

Mar 9, 2026

BiCLIP: Domain Canonicalization via Structured Geometric Transformation

Recent advances in vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities, yet adapting these models to specialized domains remains a significant challenge. Building on recent theoretical insights suggesting that independently trained VLMs are related by a canonical transformation, we extend this understanding to the concept of domains. We hypothesize that image features across disparate domains are related by a canonicalized geometric transformation that can be recovered using a small set of anchors. Few-shot classification provides a natural setting for this alignment, as the limited labeled samples serve as the anchors required to estimate this transformation. Motivated by this hypothesis, we introduce BiCLIP, a framework that applies a targeted transformation to multimodal features to enhance cross-modal alignment. Our approach is characterized by its extreme simplicity and low parameter footprint. Extensive evaluations across 11 standard benchmarks, including EuroSAT, DTD, and FGVCAircraft, demonstrate that BiCLIP consistently achieves state-of-the-art results. Furthermore, we provide empirical verification of existing geometric findings by analyzing the orthogonality and angular distribution of the learned transformations, confirming that structured alignment is the key to robust domain adaptation. Code is available at https://github.com/QuantitativeImagingLaboratory/BilinearCLIP

Pranav Mantini, Shishir K. Shah

University of Houston, Department of Computer Science · The University of Oklahoma, School of Computer Science

VLM Adaptation Domain Adaptation Vision-Language Alignment Few-Shot Domain Adaptation

Mar 7, 2026

Mind the Discriminability Trap in Source-Free Cross-domain Few-shot Learning

Source-Free Cross-Domain Few-Shot Learning (SF-CDFSL) focuses on fine-tuning with limited training data from target domains (e.g., medical or satellite images), where Vision-Language Models (VLMs) such as CLIP and SigLIP have shown promising results. Current works in traditional visual models suggest that improving visual discriminability enhances performance. However, in VLM-based SF-CDFSL tasks, we find that strengthening visual-modal discriminability actually suppresses VLMs' performance. In this paper, we aim to delve into this phenomenon for an interpretation and a solution. By both theoretical and experimental proofs, our study reveals that fine-tuning with the typical cross-entropy loss (

\mathcal{L}_{\mathrm{vlm}}

) inherently includes a visual learning part and a cross-modal learning part, where the cross-modal part is crucial for rectifying the heavily disrupted modality misalignment in SF-CDFSL. However, we find that the visual learning essentially acts as a shortcut that encourages the model to reduce

\mathcal{L}_{\mathrm{vlm}}

without considering the cross-modal part, therefore hindering the cross-modal alignment and harming the performance. Based on this interpretation, we further propose an approach to address this problem: first, we perturb the visual learning to guide the model to focus on the cross-modal alignment. Then, we use the visual-text semantic relationships to gradually align the visual and textual modalities during the fine-tuning. Extensive experiments on various settings, backbones (CLIP, SigLip, PE-Core), and tasks (4 CDFSL datasets and 11 FSL datasets) show that we consistently set new state-of-the-art results. Code is available at https://github.com/zhenyuZ-HUST/CVPR26-Mind-the-Discriminability-Trap.

Zhenyu Zhang, Yixiong Zou, Yuhua Li +2

School of Computer Science and Technology, Huazhong University of Science and Technology · National Key Laboratory for Multimedia Information Processing, Peking University

Source-Free Cross-Domain Few-Shot Learning Few-Shot Learning VLM Adaptation Few-Shot Domain Adaptation Vision-Language Alignment Source-Free Domain Adaptation

Mar 6, 2026

Self-Auditing Parameter-Efficient Fine-Tuning for Few-Shot 3D Medical Image Segmentation

Adapting foundation models to new clinical sites remains challenging in practice. Domain shift and scarce annotations must be handled by experts, yet many clinical groups do not have ready access to skilled AI engineers to tune adapter designs and training recipes. As a result, adaptation cycles can stretch from weeks to months, particularly in few-shot settings. Existing PEFT methods either require manual adapter configuration or automated searches that are computationally infeasible in few-shot 3D settings. We propose SEA-PEFT (SElf-Auditing Parameter-Efficient Fine-Tuning) to automate this process. SEA-PEFT treats adapter configuration as an online allocation problem solved during fine-tuning rather than through manual, fixed-topology choices. SEA-PEFT uses a search-audit-allocate loop that trains active adapters, estimates each adapter's Dice utility by momentarily toggling it off, and then reselects the active set under a parameter budget using a greedy knapsack allocator. Exponential Moving Average and Interquartile Range smoothing, together with a Finite-State Ranking controller, stabilize the loop and improve reliability in high-noise few-shot regimes. On TotalSegmentator and FLARE'22, SEA-PEFT improves mean Dice by 2.4--2.8 points over the strongest fixed-topology PEFT baselines across 1/5/10-shot settings while training <1% of parameters. For reproducibility purposes, we made our code publicly available at https://github.com/tsly123/SEA_PEFT

Son Thai Ly, Hien V. Nguyen

University of Houston

Parameter-Efficient Fine-Tuning Few-Shot Medical Image Segmentation 3D Medical Image Segmentation Few-Shot Domain Adaptation Medical Imaging Foundation Models

Mar 5, 2026

Reclaiming Lost Text Layers for Source-Free Cross-Domain Few-Shot Learning

Source-Free Cross-Domain Few-Shot Learning (SF-CDFSL) focuses on fine-tuning with limited training data from target domains (e.g., medical or satellite images), where CLIP has recently shown promising results due to its generalizability to downstream tasks. Current works indicate CLIP's text encoder is more suitable for cross-domain tasks, however, we find that removing certain middle layers of the text encoder can effectively improve performance in SF-CDFSL, which we call the Lost Layers. In this paper, we delve into this phenomenon for a deeper understanding. We discover that instead of being harmful for the SF-CDFSL task, the information in these layers is actually beneficial, but visual gaps prevent this useful information from being fully utilized, making these layers seem redundant. Based on this understanding, unlike current works that simply remove these layers, we propose a method to teachs the model to re-utilize information in these lost layers at both the layer and encoder levels, guiding the re-learning of the visual branch under domain shifts. Our approach effectively addresses the issue of underutilized information in the text encoder. Extensive experiments across various settings, backbones (CLIP, SigLip, PE-Core), and tasks (4 CDFSL datasets and 10 Meta-dataset datasets) demonstrate the effectiveness of our method. Code is available at https://github.com/zhenyuZ-HUST/CVPR26-VtT.

Zhenyu Zhang, Guangyao Chen, Yixiong Zou +2

School of Computer Science and Technology, Huazhong University of Science and Technology · National Key Laboratory for Multimedia Information Processing, Peking University

Source-Free Cross-Domain Few-Shot Learning Few-Shot Learning Few-Shot Domain Adaptation VLM Adaptation Source-Free Domain Adaptation Vision-Language Alignment

Feb 23, 2026

CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation

Recent vision-language models (VLMs) such as CLIP demonstrate impressive cross-modal reasoning, extending beyond images to 3D perception. Yet, these models remain fragile under domain shifts, especially when adapting from synthetic to real-world point clouds. Conventional 3D domain adaptation approaches rely on heavy trainable encoders, yielding strong accuracy but at the cost of efficiency. We introduce CLIPoint3D, the first framework for few-shot unsupervised 3D point cloud domain adaptation built upon CLIP. Our approach projects 3D samples into multiple depth maps and exploits the frozen CLIP backbone, refined through a knowledge-driven prompt tuning scheme that integrates high-level language priors with geometric cues from a lightweight 3D encoder. To adapt task-specific features effectively, we apply parameter-efficient fine-tuning to CLIP's encoders and design an entropy-guided view sampling strategy for selecting confident projections. Furthermore, an optimal transport-based alignment loss and an uncertainty-aware prototype alignment loss collaboratively bridge source-target distribution gaps while maintaining class separability. Extensive experiments on PointDA-10 and GraspNetPC-10 benchmarks show that CLIPoint3D achieves consistent 3-16% accuracy gains over both CLIP-based and conventional encoder-based baselines. Project page: https://sarthakm320.github.io/CLIPoint3D.

Mainak Singha, Sarthak Mehrotra, Paolo Casari +3

University of Trento, Italy · MDSR Labs Adobe, India · IIT Bombay, India +1

Unsupervised Domain Adaptation Few-Shot Domain Adaptation VLM Adaptation 3D VLMs Prompt Tuning

Feb 19, 2026

SCOPE: Structured Prototype-Guided Adaptation for EEG Foundation Models with Limited Labels

Electroencephalography (EEG) foundation models (EFMs) have shown strong potential for transferable representation learning, yet their adaptation in realistic settings remains challenging when only a few labeled subjects are available. We show that this challenge stems from a structural mismatch between noisy, limited supervision and the highly plastic parameter space of EFMs, reflected in three key failure modes: overconfident miscalibration, prediction collapse, and representation drift caused by unconstrained parameter updates. To address these challenges, we propose SCOPE, a Structured COnfidence-aware Prototype-guided framework for label-limited EFM adaptation. SCOPE first constructs cohort-level external supervision to provide persistent guidance and further derives confidence-aware pseudo-labels to select reliable unlabeled samples for adaptation. Building on the constructed external supervision, SCOPE introduces ProAdapter, a lightweight prototype-conditioned adapter that modulates frozen EFMs to preserve pretrained representations. Experiments across 50 label-limited adaptation settings, covering 6 EEG tasks, 5 EFM backbones, and 5%-50% training labeled-subject ratios, show that SCOPE consistently achieves strong performance and efficiency.

Jingying Ma, Feng Wu, Yucheng Xing +5

Saw Swee Hock School of Public Health, National University of Singapore, Singapore · Guangzhou Research Translation and Innovation Institute, National University of Singapore, Guangzhou, China · College of Computing and Data Science, Nanyang Technological University, Singapore +3

EEG Foundation Models Few-Shot Domain Adaptation Adapter Tuning Prototype-Based Learning

Jul 1, 2025

Few-shot Classification as Multi-instance Verification: Effective Backbone-agnostic Transfer across Domains

We investigate cross-domain few-shot learning under the constraint that fine-tuning of backbones (i.e., feature extractors) is impossible or infeasible---a scenario that is increasingly common in practical use cases---and show how representing few-shot classification as a series of multiple instance verification (MIV) tasks enables competitive classification performance using the static embeddings produced by frozen, "black-box" backbones. More specifically, based on this novel representation, we introduce a new approach to few-shot domain adaptation named the "MIV-head": a classification head that is backbone-agnostic and computationally efficient. We show that the core components of the MIV-head, when trained on few-shot data from a target domain, collectively yield strong performance on test data from that domain. Importantly, they do so without fine-tuning the backbone, requiring only the "meta-testing" phase. In experiments under various settings on an extension of the Meta-Dataset benchmark for cross-domain few-shot image classification, using representative off-the-shelf convolutional neural network and vision transformer backbones pretrained on ImageNet1K, we demonstrate that the MIV-head achieves highly competitive accuracy when compared to state-of-the-art "adapter" (or "partial fine-tuning") methods applied to the same backbones, while incurring substantially lower adaptation cost. Moreover, compared to our approach, well-known "classification head" approaches achieve a substantially lower level of accuracy. We share our code at: https://github.com/xxweka/MIV-head.

Xin Xu, Eibe Frank, Geoffrey Holmes

Department of Computer Science University of Waikato Hamilton, New Zealand

Few-Shot Domain Adaptation Few-Shot Image Classification Cross-Domain Transfer Learning

May 3, 2025

Align, Then Adapt: Few-Shot fMRI Visual Reconstruction For New Subjects

Reconstructing visual images from fMRI data is a central but challenging problem in neuroscience. Despite recent progress, current methods fall short when data and computation are limited-precisely the conditions under which this task is most critical. We introduce Adapter Alignment, a staged, architecture-agnostic training procedure for adapting a pre-trained reconstruction model to a new subject with little data. Rather than letting a shared representation space emerge implicitly, lightweight subject-specific adapters are first explicitly aligned to a pre-trained reference subject on a small set of shared calibration images, and the full pipeline is then fine-tuned while preserving this alignment. The resulting representations not only place different subjects in a shared space but also align them semantically. Our results demonstrate significant gains in low-data regimes, improving reconstruction metrics by up to 8%, and pre-diffusion retrieval shows that these gains reflect additional image-specific information in the fMRI embeddings rather than the generative prior. We also propose a coverage-based algorithm to select representative calibration images. Using both techniques together, we surpass baseline models fine-tuned with one hour of data using only 24 minutes of subject-specific data. The procedure is effective across architectures, reference subjects, and datasets, including adapting 3T subjects to a 7T reference.

Christos Zangos, Danish Ebadulla, Thomas Christopher Sprague +1

Department of Computer Science, University of California, Santa Barbara, USA · Department of Psychological and Brain Sciences, University of California, Santa Barbara, USA

fMRI-Based Visual Decoding fMRI Representation Learning Few-Shot Domain Adaptation Latent Space Alignment

Mar 5, 2025

AdaptiveCDM: Source-Free Few-Shot Domain Adaptation for Cell Detection in Microscopic Images

Cross-domain cell detection for microscopic images suffers from performance degradation due to distribution shifts across imaging domains. Unsupervised Domain Adaptation (UDA) strategies, attempt to overcome domain sift without requiring annotated data from target. However, requirement of availability of annotated data from the source domain and large-size data from target domain are both challenging limitations for realistic scenarios. This is especially true in medical imaging, where privacy requirements might prevent access to annotated source data, and costly data acquisition restricts extensive sampling of the target domain. To address these challenges, we propose AdaptiveCDM, a modular framework for Source-Free Few-Shot Domain Adaptive Object Detection (SF-FSDAOD) setting, that adapts a pretrained source model using only few labeled target images without accessing source data. AdaptiveCDM combines Resolution-Aware Augmentation (RAug) and Category-Aware Representation Learning (CARL). RAug alleviates the scarcity and class imbalance by augmenting instance balanced training examples, while preserving the scale fidelity and morphological properties of cellular structures. CARL enhances discriminative representation learning by encouraging class-consistent proposals, improving both localization and classification. We also introduce two competitive baselines for proposed setting: Faster-FreeShot and MT-FreeShot. Our approach achieves 40.4/43.4 mAP0.5 on M5 and 67.1/75.5 mAP0.5 on Raabin-WBC under 2-/5-shot adaptation. Despite using only a few labeled target images and no source data, AdaptiveCDM achieves competitive or superior performance compared with SOTA methods under their respective supervision settings. Ablations and qualitative analyses further substantiate the contribution of each component and the effectiveness of AdaptiveCDM in low-data regimes. Code/models will be available.

Nimra Dilawar, Sara Nadeem, Javed Iqbal +2

Intelligent Machines Lab, Department of Artificial Intelligence, Information Technology University, Pakistan

Source-Free Domain Adaptation Few-Shot Domain Adaptation Object Detection Microscopy Image Analysis