Query-Conditioned Prototype Adaptation for Cross-Domain Few-Shot Learning: Single-Query Inference, Controlled Comparisons, and Failure Modes

Rushab Rasik Karania; Tomas Maul

cs.CVSep 25, 2026

Query-Conditioned Prototype Adaptation for Cross-Domain Few-Shot Learning: Single-Query Inference, Controlled Comparisons, and Failure Modes

Authors: Rushab Rasik Karania, Tomas Maul

Organizations: School of Computer Science, University of Nottingham Malaysia, Semenyih, 43500, Selangor, Malaysia

Abstract

Cross-domain few-shot learning requires adapting a classifier to a new visual domain from very few labelled examples without target-time parameter updates. We isolate one question: under a fixed global representation, what does joint query-support adaptation contribute to prototype construction? The Within-Instance Prototypical Transformer (WIPT) implements single-query test-time prototype adaptation by jointly transforming one unlabelled query and the labelled support embeddings, then forming query-specific class means. Using a shared frozen ViT-S/16 encoder, miniImageNet source training, and CUB, EuroSAT and ISIC targets, we replicate the key comparisons across five independent training seeds. In 1-shot evaluation, WIPT improves frozen ProtoNet in every run on CUB (+0.21 percentage points) and EuroSAT (+2.07), but decreases ISIC (-0.22). In 5-shot evaluation, ProtoNet remains strongest overall, while WIPT consistently improves a capacity-matched support-only Transformer on ISIC (+0.99). Joint processing of up to five queries yields no reliable accuracy gain; in a head-only 5-shot benchmark, g = 5 reduces analytical attention-token pairs by 73% and peak allocated memory by 29% relative to g = 1, although latency is non-monotonic. Across all target/shot conditions, WIPT changes uncertain ProtoNet decisions far more than confident ones, and rescue/break decomposition accounts for the observed gains and losses. Source-shift and scorer controls further show that the benefit is not universal. Overall, WIPT provides a streaming-compatible form of test-time prototype adaptation that can improve difficult low-shot cross-domain decisions without target-time optimization.

Figures & tables

Figure 1: Controlled prototype construction under one shared frozen ViT-S/16 encoder. ProtoNet averages support embeddings directly. The support-only Transformer applies the same two-layer Transformer family to the support set without exposing it to the query. WIPT jointly transforms one query and the supports, then averages transformed supports class-wise to obtain query-specific prototypes. Arrows distinguish the support and query pathways; class-wise averaging follows support transformation in both Transformer methods. Classification uses negative Euclidean distance. Only the episodic Transformer heads are trained.

Shot	Domain	PN	SO	WIPT	$\Delta$ WIPT–PN	$\Delta$ WIPT–SO
5	CUB	97.88	$97.81\pm 0.03$	$97.79\pm 0.11$	$-0.09\pm 0.11$	$-0.02\pm 0.08$
5	EuroSAT	82.76	$81.46\pm 0.08$	$82.04\pm 0.84$	$-0.72\pm 0.84$	$+0.58\pm 0.79$
5	ISIC	38.92	$37.39\pm 0.49$	$38.38\pm 0.53$	$-0.54\pm 0.53$	$+0.99\pm 0.13^{\dagger}$
1	CUB	$93.26$	$\boldsymbol{93.58\pm 0.05}$	$93.47\pm 0.04$	$+0.21\pm 0.04^{\dagger}$	$-0.11\pm 0.08^{\dagger}$
1	EuroSAT	$62.09$	$63.02\pm 1.35$	$\boldsymbol{64.16\pm 0.50}$	$+2.07\pm 0.50^{\dagger}$	$+1.14\pm 0.88^{\dagger}$
1	ISIC	28.40	$28.01\pm 0.23$	$28.18\pm 0.11$	$-0.22\pm 0.11^{\dagger}$	$+0.17\pm 0.15$

Table 1: Replicated cross-domain accuracy (%) for the key controlled methods. ProtoNet (PN) is a fixed point estimate because it has no trainable episodic parameters. Support-only (SO) and WIPT are mean

\pm

sample standard deviation over five independent training seeds. Difference columns are paired training-seed mean

\pm

standard deviation. Boldface marks the highest mean accuracy within each shot/domain row.

\dagger

indicates that the corresponding unadjusted 95% training-seed interval for the difference excludes zero; no multiple-comparison correction is applied.

Figure 2: Paired cross-domain accuracy effects in 1-shot and 5-shot evaluation. PN denotes frozen ProtoNet and SO the support-only Transformer. Small points show each of the five training-seed differences; diamonds and horizontal bars show their mean and unadjusted 95% Student’s

t

interval. The zero line marks no accuracy difference. Absolute accuracies are reported in Table 1 .

Figure 3: Multi-query accuracy and compute trade-offs. Top: accuracy change relative to

g=1

in the 1-shot sparse follow-up and full 5-shot sweep, with unadjusted 95% training-seed intervals. Bottom: analytical token-pair count and recorded peak memory, each normalized to its shot-specific

g=1

benchmark. Timing benchmarks exist for

g=2,4

in 1-shot even though those groups were not included in the 1-shot accuracy study. Lower token-pair counts need not imply lower latency; full timings appear in E .

Figure 4: Decision-change decomposition and support diagnostics. Top: rescued and broken predictions per 100 queries; black diamonds show the net accuracy effect, with a 95% interval across five WIPT training seeds. The two bars have the same denominator, so their signed sum is the net effect. Bottom left: analysis-only support-to-labelled-query centroid error. This fixed-encoder quantity is identical across WIPT seeds and has no training-seed error bar. Bottom right: query-conditioned prototype variability with training-seed intervals. Both diagnostics use inter-class query-centroid distance for normalization; query labels are used only after prediction.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Trained architecture/objective variants
Variant	CUB	EuroSAT	ISIC
WIPT-2	97.864	82.324	38.753
WIPT-4	97.610	79.203	37.561
WIPT-6	97.815	81.756	38.637
Residual refinement	97.805	80.401	37.240
Combined refinement	97.845	82.185	38.419

Appendix

Table A.1: Complete cross-domain accuracy (%) for WIPT design ablations. Values are means over 1,500 matched evaluation episodes per target; WIPT-2 is the reference.

Adaptation	Scorer	CUB	EuroSAT	ISIC
Raw	Euclidean	97.880	82.763	38.918
Raw	Cosine	97.468	78.742	36.239
Raw	Relation	$79.058\pm 0.879$	$60.198\pm 2.225$	$27.983\pm 1.623$
Support-only	Euclidean	$97.810\pm 0.035$	$81.461\pm 0.104$	$37.386\pm 0.608$
Support-only	Cosine	$97.385\pm 0.019$	$77.488\pm 0.202$	$35.927\pm 0.107$
Support-only	Relation	$72.594\pm 1.743$	$58.722\pm 2.057$	$26.505\pm 0.864$

Appendix

Table B.1: Five-seed 5-shot

3\times 3

adaptation–scorer grid (accuracy, %). Trainable entries are mean

\pm

95% Student’s

t

interval across independent training seeds. Raw Euclidean and raw cosine are fixed rules and therefore have no training-seed interval.

Domain	Scorer	WIPT $-$ Raw	WIPT $-$ Support-only
CUB	Euclidean	$-0.089\pm 0.132$ (0/5)	$-0.019\pm 0.101$ (3/5)
CUB	Cosine	$-2.119\pm 0.556$ (0/5)	$-2.035\pm 0.565$ (0/5)
CUB	Relation	$+2.051\pm 2.209$ (4/5)	$+8.515\pm 2.382$ (5/5)
EuroSAT	Euclidean	$-0.721\pm 1.041$ (0/5)	$+0.581\pm 0.985$ (4/5)
EuroSAT	Cosine	$-25.317\pm 4.818$ (0/5)	$-24.063\pm 4.698$ (0/5)
EuroSAT	Relation	$-1.571\pm 3.713$ (1/5)	$-0.094\pm 3.516$ (2/5)

Appendix

Table B.2: Paired WIPT contrasts in the five-seed factorial study. Values are mean accuracy differences in percentage points

\pm

95% Student’s

t

intervals across training seeds. The parenthesized count is the number of training seeds for which WIPT has higher accuracy than the comparator.

Shot	Domain	Standard seed 0	Mixed seed 0	$\Delta$
1	CUB	93.471	93.597	+0.126
1	EuroSAT	63.921	63.670	-0.251
1	ISIC	28.139	28.059	-0.080
5	CUB	97.778	97.818	+0.040
5	EuroSAT	81.837	81.899	+0.061
5	ISIC	38.057	38.290	+0.233

Appendix

Table C.1: Single-seed mixed pseudo-domain screen.

\Delta

is mixed-shift WIPT minus the matched ordinary WIPT seed-0 accuracy (percentage points).

Figure D.1: Empirical distributions of paired episode-level normalized-margin changes for the selected 5-shot checkpoint, relative to ProtoNet (PN) and support-only (SO). Each curve uses 1,500 matched episodes. The horizontal coordinate is a difference of normalized margins, not accuracy; no training-seed uncertainty is inferred from these curves.

Figure D.2: Selected-checkpoint support-noise pathway diagnostic. The WIPT query representation from the clean-support pass is held fixed while the noisy support-derived prototypes are recomputed, isolating support-to-prototype propagation. (a) Attenuation through class-wise support averaging in each method’s respective support representation. (b) Reported margin divided by the same model’s clean margin, to avoid comparing raw distance scales. (c) Accuracy. Points are selected-checkpoint summaries; connecting lines are visual guides, and error bars are not shown.

Shot	$g$ /head	Token pairs	Fwd median	Fwd P95	F+B median	F+B P95	Peak MiB
1	PN	0	0.25	0.26	–	–	–
1	SO	50	1.25	1.36	–	–	–
1	1	5,400	1.00	1.19	5.54	5.88	44.66
1	2	3,698	2.14	2.36	9.61	9.89	48.76
1	3	3,200	1.27	1.40	5.72	6.33	48.08
1	4	3,044	2.05	2.51	9.60	9.72	46.77

Appendix

Table E.1: Head benchmark for one 5-way episode with 75 queries. PN and SO are baseline rows; numeric

g

denotes WIPT query-group size. Time is in milliseconds; memory is in MiB. F+B includes a training-mode forward pass, loss and backpropagation, without an optimizer step. P95 is the 95th percentile, not a confidence interval. Dashes indicate metrics outside the recorded benchmark coverage. Token-pair counts omit the common attention-head multiplier.

Shot	$g$	Runs summarized	Best epoch	Minutes to best
1	1	5	2 [1, 4]	–
1	3	5	4 [1, 7]	4.61 [1.16, 11.21]
1	5	5	4 [1, 7]	4.62 [1.16, 8.08]
5	1	5	3 [1, 25]	–
5	2	4	2 [1, 49]	2.70 [1.33, 67.29]
5	3	4	2 [1, 49]	2.74 [1.35, 67.71]

Appendix

Table E.2: WIPT validation-checkpoint selection. Epoch and elapsed minutes are median [minimum, maximum] over the summarized runs. The third column gives the number of runs contributing to this table, not the number of trained models: all conditions have five trained models. Dashes indicate unrecorded elapsed times.

Method	Context for adaptation	Support / prototype path	Query path and distinction
FEAT [ 7 ]	Support set	Set-to-set adaptation of support-derived class representations	Query excluded from standard prototype-adaptation attention
PrototypeFormer [ 17 ]	Within-class supports	Prototype token and support embeddings; sub-prototype contrastive training	Query embedding compared with extracted prototypes; excluded from the extraction module
CrossTransformers [ 8 ]	One query + supports	Spatially aligned support features depend on the query	Local query descriptors guide support matching; spatial rather than joint global-token adaptation
QPN [ 9 ]	One query + supports	Region-level query-specific prototypes	Local query representations guide hierarchical semantic matching
FSL-PRS [ 13 ]	Support + query set	Joint attention and pseudo-label prototype rectification	Query tokens participate in the shared attention context
APPL [ 14 ]	Support + query set	Parametric prototype learning and self-training	Multiple target queries support transductive self-training

Appendix

Table F.1: Context and representation paths of the closest methods. “Query transformation” means a change conditioned on the episode, not simply applying a learned image encoder. This is a conceptual comparison; the custom support-only control is not a reproduction of FEAT or PrototypeFormer.

Shot	$g$	CUB	EuroSAT	ISIC
5	2	$-0.015\pm 0.074$	$-0.484\pm 0.983$	$-0.141\pm 0.470$
5	3	$-0.017\pm 0.065$	$-0.459\pm 0.913$	$-0.130\pm 0.455$
5	4	$-0.015\pm 0.060$	$-0.502\pm 0.989$	$-0.135\pm 0.460$
5	5	$-0.012\pm 0.061$	$-0.500\pm 1.036$	$-0.138\pm 0.468$
1	3	$+0.039\pm 0.115$	$-0.192\pm 0.626$	$-0.103\pm 0.171$
1	5	$+0.050\pm 0.127$	$-0.159\pm 0.634$	$-0.092\pm 0.166$

Appendix

Table H.1: Multi-query accuracy change relative to single-query WIPT (

g=1

), in percentage points. Values are mean

\pm

95% Student’s

t

interval across five paired training seeds. Every interval includes zero. The 1-shot study uses the sparse follow-up

g\in\{1,3,5\}

chosen after the 5-shot sweep.

Figure I.1: Rescue and break probability versus raw ProtoNet true-class margin. For each of the six shot/domain conditions, ProtoNet errors (blue) and successes (orange) are separately sorted into ten approximately equal-count bins. Points are plotted at the median margin and show rescue probability among errors or break probability among successes. Error bars are unadjusted 95% Student’s

t

intervals across five WIPT seeds. No bin crosses the zero-margin boundary. Counts differ between the two sides; the panel annotation gives the fraction of all queries that are ProtoNet errors. Marginal accuracy contributions are shown separately in Figure 4 .

Severity	Brightness offset	Contrast multiplier	Gaussian $\sigma$
1	$+0.10$	0.75	0.08
2	$+0.20$	0.60	0.12
3	$+0.30$	0.50	0.18
4	$+0.40$	0.40	0.26
5	$+0.50$	0.30	0.38

Appendix

Table J.1: Corruption schedule. Brightness is an additive RGB offset on the

[0,1]

scale, contrast is a multiplier around the per-channel image mean, and Gaussian noise is zero-mean with standard deviation

\sigma

Figure J.1: Replicated EuroSAT corruption response across five independent 5-shot training runs. Curves show mean WIPT accuracy difference under structured (A), stochastic (B) and combined (C) corruption; error bars are 95% training-seed intervals. (a) Relative to frozen ProtoNet, effects are small and regime dependent, with combined corruption crossing in favour of WIPT at severities 4 and 5. (b) Relative to the matched support-only control, WIPT remains positive on average across the corruption study.

Shot	Domain	Standard WIPT	Shared-shift WIPT	$\Delta$ shift–standard (pp)
1	CUB	93.465	93.402	$-0.063\pm 0.099$
1	EuroSAT	64.159	63.761	$-0.399\pm 1.254$
1	ISIC	28.177	28.122	$-0.055\pm 0.302$
5	CUB	97.791	97.809	$+0.018\pm 0.144$
5	EuroSAT	82.041	81.623	$-0.419\pm 1.285$
5	ISIC	38.380	38.182	$-0.198\pm 0.655$

Appendix

Table K.1: Source-only shared-shift extension. Standard WIPT and shared-shift WIPT are means across five training seeds.

\Delta

is shared-shift WIPT minus standard WIPT with a paired 95% training-seed interval. No interval excludes zero.

Explore similar work

Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning
May 12, 2026Yaze Zhao, Yicong Liu, Yixiong Zou +2Source-Free Domain Adaptation Few-Shot Domain Adaptation
Training-Free Spectral Transductive Refinement for Cross-Domain Few-Shot Classification
Sep 20, 2026Fahim Rahman, S. M. Tanjeeb Meheran Rohan, Md. Taimum Ibne Sayed +2Few-Shot Learning Spectral Methods
Department of Computer Science and Engineering Islamic University of Technology Gazipur, Bangladesh
Learning Multi-Modal Prototypes for Cross-Domain Few-Shot Object Detection
Feb 21, 2026Wanqi Wang, Jingcai Guo, Yuxiang Cai +1Few-Shot Object Detection Object Detection
University of Chinese Academy of Sciences · The Hong Kong Polytechnic University · Zhejiang University +1

May 12, 2026J/K move · Enter open · S save

Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning

Yaze Zhao, Yicong Liu, Yixiong Zou, Yuhua Li+1

Source-Free Domain Adaptation Few-Shot Domain Adaptation VLM Adaptation Few-Shot Learning Vision-Language Alignment Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Cross-Domain Few-Shot Learning (CDFSL) aims to adapt large-scale pretrained models to specialized target domains with limited samples, yet the few-shot fine-tuning of vision-language models like CLIP remains underexplored. By establishing multiple fine-tuning baselines of CLIP for CDFSL, we find adapter-based methods (e.g., LoRA) consistently outperform prompt-based ones (e.g., MaPLe), contrary to in-domain scenarios. To make those effective in-domain methods competitive again in CDFSL, we analyze this phenomenon and discover LoRA's superiority stems from rectifying the collapsed attention of visual CLS token, enhancing modality alignment and class separation by focusing on text-related visual regions. Further, we find textual EOS token exhibit much better attention to visual samples, and CLIP's standard contrastive loss weakly constrains modality alignment. Based on these insights, we propose Semantic Probe, a plug-and-play attention rectification framework for both adapter- and prompt-based methods. Extensive experiments on four CDFSL benchmarks validate our rationale, achieving state-of-the-art performance and benefiting both fine-tuning paradigms. Codes will be released.

Abstract

Explore similar work

Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning

Training-Free Spectral Transductive Refinement for Cross-Domain Few-Shot Classification

Learning Multi-Modal Prototypes for Cross-Domain Few-Shot Object Detection