Few-Shot Learning

A key challenge in Few-Shot Recognition (FSR) is the scarcity of labeled task-specific data for training. Contemporary methods mitigate this challenge by exploiting pretrained Vision-Language Models (VLMs) for their learned transferable features. Some recent ones augment the scarce data by retrieving relevant yet out-of-distribution (OOD) examples from external data sources. However, they largely overlook an equally fundamental challenge: the absence of validation data for model selection and hyperparameter tuning. As a result, the adapted VLMs overfit the few-shot training data and generalize poorly to OOD test data. We show that retrieved data provide a natural source of validation, but their direct use creates a fundamental paradox. Since the retrieved samples are inherently OOD w.r.t the few-shot ID training data, adapting a VLM to the ID data constantly degrades its performance on the retrieved data. Consequently, conventional validation relying on the accuracy of retrieved data favors no adaptation and prevents improved generalization. To resolve this dilemma, we formulate validation as a multi-objective optimization (MOO) problem between performance gains on the few-shot ID data and the performance preservation on the retrieved data. We introduce a simple yet effective validation strategy by choosing an operating point on the Pareto front of the MOO problem. This strategy enables hyperparameter tuning, which had been unrealistically done by using test data, thereby mitigating overfitting and improving generalization. We integrate it into a stage-wise finetuning pipeline, termed Validation-Enabled Stage-wise Tuning (VEST). Extensive experiments on five established benchmarks show that VEST consistently outperforms existing approaches on both ID and OOD test data.

Few-shot action recognition (FSAR) aims to recognize novel action categories with few exemplars. Existing methods typically learn frame-level representations for each video by designing inter-frame temporal modeling strategies or inter-video interaction at the coarse video-level granularity. However, they treat each episode task in isolation and neglect fine-grained temporal relation modeling between videos, thus failing to capture shared fine-grained temporal patterns across videos and reuse temporal knowledge from historical tasks. In light of this, we propose HR2G-shot, a Hierarchical Relation-augmented Representation Generalization framework for FSAR, which unifies three types of relation modeling (inter-frame, inter-video, and inter-task) to learn task-specific temporal patterns from a holistic view. Going beyond conducting inter-frame temporal interactions, we further devise two components to respectively explore inter-video and inter-task relationships: i) Inter-video Semantic Correlation (ISC) performs cross-video frame-level interactions in a fine-grained manner, thereby capturing task-specific query features and enhancing both intra-class consistency and inter-class separability; ii) Inter-task Knowledge Transfer (IKT) retrieves and aggregates relevant temporal knowledge from the bank, which stores diverse temporal patterns from historical episode tasks. Extensive experiments on five benchmarks show that HR2G-shot outperforms current top-leading FSAR methods.

Meta-learning represents a strong class of approaches for solving few-shot learning tasks. Nonetheless, recent research suggests that simply pre-training a generic encoder can potentially surpass meta-learning algorithms. In this paper, we first discuss the reasons why meta-learning fails to stand out in these few-shot learning experiments, and hypothesize that it is due to the few-shot learning tasks lacking diversity. We propose DRESS, a task-agnostic Disentangled REpresentation-based Self-Supervised meta-learning approach that enables fast model adaptation on highly diversified few-shot learning tasks. Specifically, DRESS utilizes disentangled representation learning to create self-supervised tasks that can fuel the meta-training process. Furthermore, we also propose a class-partition based metric for quantifying the task diversity directly on the input space. We validate the effectiveness of DRESS through experiments on datasets with multiple factors of variation and varying complexity. The results suggest that DRESS is able to outperform competing methods on the majority of the datasets and task setups. Through this paper, we advocate for a re-examination of proper setups for task adaptation studies, and aim to reignite interest in the potential of meta-learning for solving few-shot learning tasks via disentangled representations.

Instruction tuning aligns large language models (LLMs) with human intentions but requires diverse, high-quality data that are difficult to collect in privacy-sensitive domains. Federated instruction tuning (FedIT) enables collaborative training across data owners, yet existing methods typically assume sufficient local data. In realistic few-shot settings, limited samples can cause overfitting, degrade performance, and increase vulnerability to training data extraction attacks. We propose PPFedIT, a federated algorithm that improves both model performance and privacy protection in federated few-shot learning. It comprises three client-side steps: (1) synthetic data generation, which uses LLMs to diversify and enrich local data; (2) parameter isolation training, which updates the shared global LLM on synthetic data and local LLMs on private local data to mitigate synthetic-data noise; and (3) local aggregation then sharing, which mixes global and local model parameters before uploading them for server aggregation to mitigate data extraction attacks. Experiments on three open-source datasets show that PPFedIT improves model performance by an average of 8.4% and reduces the risk of data extraction attacks by approximately 20% in challenging federated few-shot settings.

Medical landmark detection usually requires accurate expert annotations, which are laborious and difficult to scale across anatomical regions. In this work, we study an extreme annotation-efficient setting where only a single annotated template image is available. We propose SGB-Match, a structure-guided coarse-to-fine self-supervised matching framework for one-shot medical landmark detection. The framework first learns dense anatomical correspondence from unlabeled augmented image pairs, and then transfers the landmark definition from the annotated template to each target image through feature matching. Different from standard contrastive correspondence learning, where negative candidates are penalized by a structure-agnostic rule, we introduce a structure-guided bias into the contrastive objective. The bias is constructed from relative distance and edge-aware anatomical cues, and explicitly reweights the negative gradients: nearby structure-relevant candidates are weakly repelled, while distant or structure-irrelevant negatives are strongly suppressed. As a result, the learned feature space better preserves local anatomical structures around template landmarks and reduces confusing responses from repeated textures. We further adopt a global-to-local design, where a global encoder provides coarse landmark localization and a local encoder refines the prediction in a cropped region. Extensive experiments on four 2D radiological landmark datasets demonstrate that SGB-Match achieves strong one-shot performance across both public and newly collected datasets and consistently benefits from both structure-guided bias and two-stage refinement.

After learning a multiple paired-association (MPA) navigation task, rodents can rapidly learn new cue-location associations after a single exposure, a phenomenon proposed to be facilitated by cognitive schemas. However, it remains unclear what schemas support this behavior and how they could be implemented in biological neural circuits. Here, we identify a set of schemas sufficient for gradual then one-shot paired-association navigation, and implement them in a fully neural agent using biologically plausible plasticity rules. The agent gradually acquires a low-dimensional spatial metric representation using a path-integration based temporal-difference error, enabling self-localization across different environments. It also contains a memory network that forms one-shot associations between sensory cues and goal locations, implemented using reward-modulated exploratory Hebbian (EH) plasticity. A third network performs vector subtraction between the agent's current and goal location to compute goal-directed actions. Supplementing these schemas with an actor-critic enables one-shot navigation past obstacles. Lastly, learning a working-memory gating mechanism using the temporal-difference error enables one-shot learning even with brief cue and distractor presentation. The model reproduces key behavioral features of rodent learning in the MPA task and generates testable predictions about the neural mechanisms underlying schema-based one-shot learning.