LE-GDA: Layer Ensemble Gaussian Discriminant Analysis for Training-Free Few-Shot CLIP Adaptation

Pavodi Maniamfu; Vogel Kiketa; Zezina Kuzulu Kuetu; Jakin Mukendi Kabongo; David Mayoya Kutangila; Pescie Honore Kinkani Mampuya; L'Or Wolo Sieka

cs.CVOct 26, 2026

LE-GDA: Layer Ensemble Gaussian Discriminant Analysis for Training-Free Few-Shot CLIP Adaptation

Authors: Pavodi Maniamfu, Vogel Kiketa, Zezina Kuzulu Kuetu, Jakin Mukendi Kabongo, David Mayoya Kutangila, Pescie Honore Kinkani Mampuya, L'Or Wolo Sieka

Organizations: Temple University Japan Campus · Universit´e de Kinshasa · Universit´e Protestante au Congo · Handong Global University ·

Abstract

Training-free few-shot adaptation of vision-language models typically constructs a classifier from CLIP's final projected image representation, leaving intermediate representations unused. Combining them is not straightforward. Class scores from different representations differ in spread by orders of magnitude for the same query. Direct averaging allocates influence by numerical scale rather than predictive information. We propose Layer-Ensemble Gaussian Discriminant Analysis (LE-GDA), a method that fits regularized shared-covariance discriminants independently to intermediate and final representations extracted in one frozen encoder pass. LE-GDA standardizes each discriminant's class scores per query, selects a small representation multiset on labeled validation data, and combines it with the zero-shot classifier without gradient-based optimization. Across 11 datasets, four CLIP backbones, five shot counts, and three independently sampled K-shot training sets, LE-GDA achieves the highest mean accuracy of all evaluated methods on every backbone, raising the 16-shot mean from 78.01% to 79.54% over a matched GDA-CLIP (final-feature) baseline and improving that baseline for every backbone-shot combination. At one shot, LE-GDA retains a 1.20-point mean gain over the baseline even though covariance estimation is unavailable and per-query standardization is neutral. A control with richer per-class structure inside a single representation does not reproduce the gain, indicating that the benefit comes from combining representations. These results show that intermediate CLIP representations carry information beyond the final embedding and can improve training-free few-shot adaptation. Code and evaluation resources are available at https://github.com/pavodi-NM/le-gda.

Figures & tables

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	–	58.81	59.59	62.11	65.43	61.49
Tip-Adapter	–	orig.	70.20 $\pm$ .13	72.06 $\pm$ .12	72.88 $\pm$ .16	76.88 $\pm$ .02	73.01
Proto-Adapter	–	ours	72.48 $\pm$ .19	73.64 $\pm$ .14	74.52 $\pm$ .03	78.34 $\pm$ .06	74.74
GPCache (W+C)	–	ours	72.81 $\pm$ .05	75.26 $\pm$ .21	75.61 $\pm$ .10	79.42 $\pm$ .14	75.78
Tip-Adapter-F	✓	orig.	75.50 $\pm$ .14	77.49 $\pm$ .04	77.61 $\pm$ .03	81.16 $\pm$ .08	77.94
Proto-Adapter-F	✓	ours	75.39 $\pm$ .10	76.67 $\pm$ .09	76.76 $\pm$ .02	80.39 $\pm$ .06	77.30

Table 1: Mean top-1 accuracy (%) over 11 datasets at

K=16

. “Grad.” indicates gradient-based adaptation. “Impl.” indicates whether the row was produced by the authors’ released code (orig.) or by our implementation (ours), in both cases under our protocol (Sec. 4 ). Bold denotes the best result in each accuracy column.

Method	Caltech	DTD	EuroSAT	Aircraft	Food	ImageNet	Flowers	Pets	Cars	SUN	UCF	Mean
GDA-CLIP (final-feature)	93.76	69.03	84.91	42.21	82.51	66.47	96.08	91.33	78.97	72.94	79.95	78.01
LE-GDA (ours)	94.89	69.71	89.77	46.90	82.48	66.63	97.28	91.17	79.97	73.84	82.28	79.54
$\Delta$ (LE-GDA)	+1.13	+0.68	+4.86	+4.69	$-$ 0.03	+0.16	+1.20	$-$ 0.16	+1.00	+0.90	+2.33	+1.52
GDA-CLIP (published)	94.10	68.51	87.23	43.82	82.58	–	96.49	91.30	78.82	73.06	80.28	79.62 †
LE-GDA (ours)	94.89	69.71	89.77	46.90	82.48	–	97.28	91.17	79.97	73.84	82.28	80.83 †
$\Delta$ (LE-GDA)	+0.79	+1.20	+2.54	+3.08	$-$ 0.10	–	+0.79	$-$ 0.13	+1.15	+0.78	+2.00	+1.21 †

Table 2: Top-1 accuracy (%) at

K=16

, averaged over four backbones. Upper block: matched implementations under our protocol ( Sec. 4 ). Lower block: published GDA-CLIP figures [ 32 ] , included as an external consistency check rather than a matched comparison. The LE-GDA row repeats the numbers above. Bold denotes the best result per dataset within each block. Means are over 11 datasets in the upper block and, marked † , over 10 in the lower block, where ImageNet is excluded because our evaluation uses a disjoint 40,000-image partition; the two are therefore not directly comparable.

Backbone	$K=1$	$K=2$	$K=4$	$K=8$	$K=16$
RN50	+1.24	+1.40	+1.28	+1.57	+1.91
RN101	+1.37	+1.19	+1.51	+1.70	+2.27
ViT-B/32	+0.80	+0.80	+0.44	+0.84	+1.00
ViT-B/16	+1.40	+0.73	+0.75	+0.93	+0.91
Mean	+1.20	+1.03	+1.00	+1.26	+1.52

Table 3: LE-GDA improvement over the matched GDA-CLIP (final-feature) baseline (accuracy points), averaged over 11 datasets.

Panel of Figure 1 — Figure 1: Few-shot performance as the number of labeled examples per class is varied. Results are averaged over 11 datasets in both panels. The GDA-CLIP label in the panels denotes the matched GDA-CLIP (final-feature) implementation described in Sec. 4 . LE-GDA improves this baseline for every backbone and evaluated shot count (b).

Comparison	Median $\Delta$
Matched search control vs. GDA-CLIP (final-feature)	+0.000
LE-GDA vs. GDA-CLIP (final-feature)	+0.829

Table 4: Median paired accuracy differences from GDA-CLIP (final-feature) at

K=16

, over 11 datasets, four backbones, and three seeds.

Selected component	Mean $\Delta\pm$ sd	Share
Feature-space multiset	$+0.941\pm 1.269$	41%
Fusion coefficient $\alpha$	$+1.046\pm 1.487$	46%
Ridge coefficient $\lambda$	$+0.283\pm 0.875$	$\sim$ 13%

Table 5: Decomposition of the

2.270

-point difference between fully validation-selected and fully transferred LE-GDA at

K=16

. Differences are paired over 11 datasets and four backbones.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Backbone	Method	Caltech	DTD	EuroSAT	Aircraft	Food	ImageNet	Flowers	Pets	Cars	SUN	UCF	Mean
RN50	GDA-CLIP (final-feature)	91.93	67.18	83.49	39.06	79.02	62.37	95.44	89.53	74.83	70.45	77.86	75.56
	LE-GDA	93.67	68.18	88.97	44.45	79.07	62.82	97.09	89.23	76.55	71.49	80.67	77.47
RN101	GDA-CLIP (final-feature)	93.74	68.64	83.02	40.23	82.10	65.41	95.57	91.24	80.52	72.10	78.94	77.41
	LE-GDA	95.09	70.39	89.22	48.06	82.15	65.88	97.77	90.95	82.03	73.22	81.67	79.68
ViT-B/32	GDA-CLIP (final-feature)	94.27	67.79	85.62	40.64	81.76	66.67	95.95	90.82	77.51	73.46	80.14	77.69
	LE-GDA	94.97	68.16	90.34	43.15	81.58	66.46	96.35	90.86	77.94	74.14	81.66	78.69

Appendix

Table S1 : Top-1 accuracy (%) for every dataset and CLIP backbone at

K=16

, for LE-GDA and its matched baseline only. Results are averaged over three independently sampled

K

-shot training sets. Tables S2 – S12 give the same settings for all eight methods.

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	85.72 $\pm$ .00	89.98 $\pm$ .00	91.48 $\pm$ .00	93.27 $\pm$ .00	90.11
Tip-Adapter	–	orig.	90.33 $\pm$ .12	92.95 $\pm$ .13	93.81 $\pm$ .37	94.86 $\pm$ .19	92.99
Proto-Adapter	–	ours	91.63 $\pm$ .07	93.37 $\pm$ .22	93.78 $\pm$ .30	95.36 $\pm$ .23	93.54
GPCache (W+C)	–	ours	91.62 $\pm$ .12	93.63 $\pm$ .22	94.37 $\pm$ .18	95.36 $\pm$ .14	93.75
Tip-Adapter-F	✓	orig.	92.45 $\pm$ .21	94.55 $\pm$ .05	95.29 $\pm$ .22	95.81 $\pm$ .11	94.53
Proto-Adapter-F	✓	ours	92.52 $\pm$ .23	94.16 $\pm$ .40	94.86 $\pm$ .16	95.66 $\pm$ .43	94.30

Appendix

Table S2 : Per-method top-1 accuracy (%) on Caltech101 at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	42.97 $\pm$ .00	38.12 $\pm$ .00	44.15 $\pm$ .00	44.03 $\pm$ .00	42.32
Tip-Adapter	–	orig.	60.66 $\pm$ .17	63.04 $\pm$ .36	62.61 $\pm$ .42	65.54 $\pm$ .39	62.96
Proto-Adapter	–	ours	65.01 $\pm$ .34	64.81 $\pm$ .44	65.35 $\pm$ .41	67.45 $\pm$ .74	65.66
GPCache (W+C)	–	ours	64.76 $\pm$ .62	67.42 $\pm$ .70	66.35 $\pm$ .64	69.94 $\pm$ .28	67.11
Tip-Adapter-F	✓	orig.	66.21 $\pm$ .23	69.29 $\pm$ .06	69.13 $\pm$ .84	72.16 $\pm$ .48	69.20
Proto-Adapter-F	✓	ours	67.63 $\pm$ .78	68.09 $\pm$ .60	67.34 $\pm$ .52	70.41 $\pm$ .66	68.37

Appendix

Table S3 : Per-method top-1 accuracy (%) on DTD at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	36.27 $\pm$ .00	32.78 $\pm$ .00	45.31 $\pm$ .00	48.49 $\pm$ .00	40.71
Tip-Adapter	–	orig.	71.67 $\pm$ 1.24	68.47 $\pm$ 1.90	74.14 $\pm$ .90	77.87 $\pm$ .35	73.04
Proto-Adapter	–	ours	74.84 $\pm$ 1.10	69.22 $\pm$ 1.20	75.93 $\pm$ .54	79.10 $\pm$ .34	74.77
GPCache (W+C)	–	ours	77.60 $\pm$ 1.11	76.31 $\pm$ .49	80.76 $\pm$ .25	83.26 $\pm$ .70	79.48
Tip-Adapter-F	✓	orig.	85.28 $\pm$ .82	82.89 $\pm$ .90	85.87 $\pm$ .15	87.56 $\pm$ .08	85.40
Proto-Adapter-F	✓	ours	80.08 $\pm$ .83	74.66 $\pm$ .96	79.78 $\pm$ .47	81.15 $\pm$ .82	78.92

Appendix

Table S4 : Per-method top-1 accuracy (%) on EuroSAT at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	17.10 $\pm$ .00	17.91 $\pm$ .00	19.08 $\pm$ .00	24.81 $\pm$ .00	19.73
Tip-Adapter	–	orig.	29.61 $\pm$ .43	31.73 $\pm$ .40	31.41 $\pm$ .89	40.46 $\pm$ .48	33.31
Proto-Adapter	–	ours	27.63 $\pm$ .28	29.01 $\pm$ .36	31.17 $\pm$ .30	38.83 $\pm$ .57	31.66
GPCache (W+C)	–	ours	32.48 $\pm$ .20	35.96 $\pm$ .81	35.48 $\pm$ .09	43.91 $\pm$ .37	36.96
Tip-Adapter-F	✓	orig.	35.40 $\pm$ .37	37.66 $\pm$ .67	36.69 $\pm$ .21	45.75 $\pm$ .32	38.88
Proto-Adapter-F	✓	ours	35.79 $\pm$ .17	37.55 $\pm$ .22	36.87 $\pm$ .38	45.42 $\pm$ .51	38.91

Appendix

Table S5 : Per-method top-1 accuracy (%) on FGVC-Aircraft at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	77.38 $\pm$ .00	80.68 $\pm$ .00	80.41 $\pm$ .00	85.88 $\pm$ .00	81.09
Tip-Adapter	–	orig.	77.85 $\pm$ .13	81.05 $\pm$ .06	80.89 $\pm$ .05	86.36 $\pm$ .12	81.54
Proto-Adapter	–	ours	78.62 $\pm$ .06	81.74 $\pm$ .10	81.25 $\pm$ .10	86.93 $\pm$ .04	82.13
GPCache (W+C)	–	ours	78.29 $\pm$ .10	81.73 $\pm$ .09	81.30 $\pm$ .09	86.94 $\pm$ .04	82.06
Tip-Adapter-F	✓	orig.	79.33 $\pm$ .14	82.22 $\pm$ .26	82.16 $\pm$ .06	87.30 $\pm$ .07	82.75
Proto-Adapter-F	✓	ours	79.01 $\pm$ .04	82.10 $\pm$ .12	81.64 $\pm$ .11	87.12 $\pm$ .15	82.47

Appendix

Table S6 : Per-method top-1 accuracy (%) on Food101 at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	59.15 $\pm$ .00	61.25 $\pm$ .00	63.13 $\pm$ .00	67.84 $\pm$ .00	62.84
Tip-Adapter	–	orig.	60.39 $\pm$ .06	63.10 $\pm$ .11	64.70 $\pm$ .05	69.47 $\pm$ .08	64.42
Proto-Adapter	–	ours	61.97 $\pm$ .09	64.85 $\pm$ .16	65.90 $\pm$ .05	70.77 $\pm$ .08	65.87
GPCache (W+C)	–	ours	61.50 $\pm$ .10	64.47 $\pm$ .05	65.73 $\pm$ .10	70.58 $\pm$ .09	65.57
Tip-Adapter-F	✓	orig.	63.29 $\pm$ .03	66.36 $\pm$ .03	67.33 $\pm$ .07	71.98 $\pm$ .07	67.24
Proto-Adapter-F	✓	ours	63.42 $\pm$ .03	66.28 $\pm$ .08	67.49 $\pm$ .08	72.19 $\pm$ .05	67.35

Appendix

Table S7 : Per-method top-1 accuracy (%) on ImageNet at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	66.06 $\pm$ .00	64.43 $\pm$ .00	66.50 $\pm$ .00	70.81 $\pm$ .00	66.95
Tip-Adapter	–	orig.	89.39 $\pm$ .32	90.19 $\pm$ .30	91.18 $\pm$ .07	93.90 $\pm$ .07	91.16
Proto-Adapter	–	ours	92.11 $\pm$ .18	92.34 $\pm$ .15	92.66 $\pm$ .13	95.60 $\pm$ .08	93.18
GPCache (W+C)	–	ours	92.72 $\pm$ .12	94.68 $\pm$ .17	94.30 $\pm$ .38	96.91 $\pm$ .29	94.65
Tip-Adapter-F	✓	orig.	94.56 $\pm$ .58	94.92 $\pm$ .10	93.99 $\pm$ .26	96.52 $\pm$ .32	95.00
Proto-Adapter-F	✓	ours	95.55 $\pm$ .05	95.82 $\pm$ .25	95.28 $\pm$ .02	97.28 $\pm$ .03	95.98

Appendix

Table S8 : Per-method top-1 accuracy (%) on Flowers102 at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	85.80 $\pm$ .00	86.94 $\pm$ .00	87.44 $\pm$ .00	89.10 $\pm$ .00	87.32
Tip-Adapter	–	orig.	87.90 $\pm$ .25	87.65 $\pm$ .10	88.27 $\pm$ .57	91.74 $\pm$ .50	88.89
Proto-Adapter	–	ours	88.23 $\pm$ .19	89.21 $\pm$ .40	89.56 $\pm$ .21	92.15 $\pm$ .17	89.79
GPCache (W+C)	–	ours	87.97 $\pm$ .06	88.90 $\pm$ .53	89.32 $\pm$ .75	92.40 $\pm$ .34	89.65
Tip-Adapter-F	✓	orig.	89.30 $\pm$ .19	90.39 $\pm$ .22	90.72 $\pm$ .32	92.96 $\pm$ .29	90.84
Proto-Adapter-F	✓	ours	88.94 $\pm$ .45	90.88 $\pm$ .32	89.73 $\pm$ .08	92.80 $\pm$ .23	90.59

Appendix

Table S9 : Per-method top-1 accuracy (%) on OxfordPets at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	55.73 $\pm$ .00	63.21 $\pm$ .00	60.20 $\pm$ .00	65.55 $\pm$ .00	61.17
Tip-Adapter	–	orig.	66.14 $\pm$ .39	72.71 $\pm$ .18	70.01 $\pm$ .33	75.52 $\pm$ .31	71.10
Proto-Adapter	–	ours	70.32 $\pm$ .19	76.24 $\pm$ .56	72.81 $\pm$ .11	79.20 $\pm$ .29	74.64
GPCache (W+C)	–	ours	69.59 $\pm$ .60	76.79 $\pm$ .39	73.25 $\pm$ .12	78.59 $\pm$ .11	74.56
Tip-Adapter-F	✓	orig.	74.92 $\pm$ .56	80.67 $\pm$ .39	77.88 $\pm$ .59	82.76 $\pm$ .06	79.06
Proto-Adapter-F	✓	ours	75.97 $\pm$ .51	80.81 $\pm$ .41	77.10 $\pm$ .40	82.78 $\pm$ .48	79.16

Appendix

Table S10 : Per-method top-1 accuracy (%) on StanfordCars at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	58.85 $\pm$ .00	59.02 $\pm$ .00	62.06 $\pm$ .00	62.56 $\pm$ .00	60.62
Tip-Adapter	–	orig.	67.02 $\pm$ .24	68.19 $\pm$ .19	70.26 $\pm$ .22	71.71 $\pm$ .22	69.30
Proto-Adapter	–	ours	70.09 $\pm$ .15	71.05 $\pm$ .15	72.80 $\pm$ .06	74.62 $\pm$ .30	72.14
GPCache (W+C)	–	ours	69.24 $\pm$ .14	70.99 $\pm$ .20	72.63 $\pm$ .08	74.38 $\pm$ .36	71.81
Tip-Adapter-F	✓	orig.	71.29 $\pm$ .05	72.83 $\pm$ .17	74.02 $\pm$ .22	76.17 $\pm$ .16	73.58
Proto-Adapter-F	✓	ours	71.61 $\pm$ .13	72.94 $\pm$ .09	74.12 $\pm$ .14	76.16 $\pm$ .25	73.71

Appendix

Table S11 : Per-method top-1 accuracy (%) on SUN397 at

K=16

Method	Grad.	Impl.	RN50	RN101	ViT-B/32	ViT-B/16	Mean
Zero-shot CLIP	–	—	61.86 $\pm$ .00	61.12 $\pm$ .00	63.49 $\pm$ .00	67.41 $\pm$ .00	63.47
Tip-Adapter	–	orig.	71.27 $\pm$ .24	73.52 $\pm$ .33	74.39 $\pm$ .30	78.26 $\pm$ .22	74.36
Proto-Adapter	–	ours	76.80 $\pm$ .95	78.20 $\pm$ .28	78.46 $\pm$ .43	81.69 $\pm$ .23	78.79
GPCache (W+C)	–	ours	75.12 $\pm$ .11	77.01 $\pm$ .31	78.21 $\pm$ .25	81.32 $\pm$ .22	77.91
Tip-Adapter-F	✓	orig.	78.52 $\pm$ .79	80.66 $\pm$ .35	80.66 $\pm$ .54	83.79 $\pm$ .17	80.91
Proto-Adapter-F	✓	ours	78.76 $\pm$ .84	80.08 $\pm$ .36	80.17 $\pm$ .30	83.35 $\pm$ .08	80.59

Appendix

Table S12 : Per-method top-1 accuracy (%) on UCF101 at

K=16

Backbone	$O_{\rm sel.}$	$O_{\rm ctrl.}$	$H_{\rm sel.}$	$H_{\rm ctrl.}$	Distinct spaces
ViT-B/16	4.87	6.90	0.0525	0.0221	2.64
ViT-B/32	4.09	6.61	0.0578	0.0242	2.67
RN50	4.08	4.52	0.0764	0.0415	2.55
RN101	4.80	5.56	0.0746	0.0327	2.52

Appendix

Table S15 : Statistics

O

and

H

K=16

O

is the median ratio of joint error probability to the product of the two marginal error probabilities, and

H

is the fraction of test examples on which at least one ensemble classifier is correct while GDA-CLIP (final-feature) is wrong. Results are averaged over 11 datasets and three random seeds. “Selected” denotes the LE-GDA multiset, and “control” denotes the matched search control, an ensemble containing the same number of GDA-CLIP (final-feature) classifiers with different ridge coefficients.

Components	Mean accuracy	$\Delta$ vs. $m=1$
$m=1$	80.85	0.000
$m=2$	80.75	$-0.100$
$m=4$	80.47	$-0.378$
$m=8$	79.75	$-1.097$
$m=16$	78.75	$-2.098$
Validation-selected $m$	80.87	+0.020

Appendix

Table S16 : Gaussian-mixture ablation at

K=16

over 10 datasets, four backbones, and three independently sampled

K

-shot training sets; ImageNet is excluded. Mean accuracy is averaged over the 10 datasets and four backbones. The shared covariance is estimated once per feature space and held fixed across

m

. Validation selection searches

m\in\{1,2,4,8\}

;

m=16

is evaluated only as a fixed setting. The

m=1

estimator is identical to LE-GDA.

Comparison	Median $\Delta$	Positive	Negative	Tied
LE-GDA	$+0.829$	109	22	1
Matched search control	$+0.000$	52	54	26
Per-query score standardization	$+0.373$	98	31	3
Within-query score ranks	$-0.628$	25	93	14
Median across feature spaces	$-0.122$	13	77	42
Mean softmax probabilities	$-0.069$	52	67	13

Appendix

Table S17 : Paired comparisons at

K=16

over 11 datasets, four backbones, and three random seeds. The first two rows use GDA-CLIP (final-feature) as the baseline; the remaining rows use uniform averaging of globally scaled scores. Zero differences are included in the medians.

Comparison	Feature	$d$	Stand. $\Delta$	Unif.	Std.
Dispersion matched	RN50 stage 1	$4.0\!\times\!10^{-5}$	$-8.09$	10	0
Dispersion matched	RN50 stage 3	$6.3\!\times\!10^{-5}$	$-0.56$	14	44
Accuracy matched	RN101 stage 3	$1.3\!\times\!10^{-4}$	$+0.48$	24	46
Accuracy matched	RN101 final	$3.2\!\times\!10^{-1}$	$0.00$	32	47

Appendix

Table S18 : Controlled comparisons separating score dispersion from standalone accuracy at

K=16

. Here

d

is the median validation-set dispersion of the class scores within a query, and Standalone

\Delta

is the top-1 accuracy difference relative to matched GDA-CLIP (final-feature) [ 32 ] . Counts include repeated selections over 33 dataset–run combinations per backbone and may exceed 33 because greedy ensemble selection uses replacement.

Panel of Figure 23 — Figure S1 : Per-query standardized selection counts and standalone top-1 accuracy differences (points) relative to the final embedding for RN50 at $K=16$ . Panel (a) shows greedy selections; panel (b) shows standalone accuracy differences. Dataset rows are ordered by LE-GDA gain averaged over the four backbones. Counts are over three runs and include repeated selections; empty cells denote spaces never selected. Values in parentheses are the gains in Tab. 2. Colors in (b) are clipped at $\pm 3$ points; ref. marks the final embedding.

Panel of Figure 24 — Figure S3 : Per-query standardized selection counts and standalone top-1 accuracy differences (points) relative to the final embedding for ViT-B/16 at $K=16$ . Panel (a) shows greedy selections; panel (b) shows standalone accuracy differences. Dataset rows are ordered by LE-GDA gain averaged over the four backbones. Counts are over three runs and include repeated selections; empty cells denote spaces never selected. Values in parentheses are the gains in Tab. 2. Colors in (b) are clipped at $\pm 3$ points; ref. marks the final embedding.

Configuration	Representation multiset	Ridge coefficient $\lambda$	Fusion coefficient $\alpha$	Held-out validation used?
0	none (zero-shot CLIP)	none	none	no
1	final representation	closed-form Ledoit–Wolf	transferred	no
2	transferred	validation-selected	validation-selected	yes
3	transferred	transferred	transferred	no
3a	transferred	transferred	validation-selected	yes
4	validation-selected	validation-selected	validation-selected	yes

Appendix

Table S19 : Configurations used in the transfer experiment. “Transferred” denotes a leave-one-dataset-out value derived from the other ten datasets.

Backbone	Config. 0	Config. 1	Config. 2	Config. 3	Config. 3a	Config. 4	$\Delta(3-1)$	$\Delta(4-3)$
ViT-B/16	65.43 $\pm$ 20.48	80.29 $\pm$ 15.16	81.12 $\pm$ 14.45	80.38 $\pm$ 14.43	81.05 $\pm$ 14.44	82.31 $\pm$ 13.55	+0.09 $\pm$ 1.90	+1.93 $\pm$ 1.86
ViT-B/32	62.11 $\pm$ 20.79	76.42 $\pm$ 16.85	77.72 $\pm$ 16.20	76.80 $\pm$ 17.10	77.54 $\pm$ 16.48	78.69 $\pm$ 15.56	+0.39 $\pm$ 1.35	+1.89 $\pm$ 2.34
RN50	58.81 $\pm$ 20.90	73.96 $\pm$ 16.30	76.80 $\pm$ 15.56	75.15 $\pm$ 15.95	76.53 $\pm$ 15.96	77.47 $\pm$ 15.36	+1.19 $\pm$ 3.52	+2.32 $\pm$ 2.16
RN101	59.59 $\pm$ 22.53	75.96 $\pm$ 16.59	78.74 $\pm$ 14.67	76.74 $\pm$ 16.16	78.14 $\pm$ 15.84	79.68 $\pm$ 14.55	+0.78 $\pm$ 3.80	+2.94 $\pm$ 2.81

Appendix

Table S20 : Configuration-transfer results at

K=16

. Entries for Configurations 0, 1, 2, 3, 3a, and 4 are mean accuracy

\pm

sample standard deviation across the 11 datasets. The final columns are paired differences between the validation-free configurations and between fully validation-selected and fully transferred LE-GDA.

Backbone	Config. 0	Config. 1	Config. 2	Config. 3	Config. 3a	Config. 4	$\Delta(3-1)$	$\Delta(4-3)$
ViT-B/16	65.43 $\pm$ 20.48	74.78 $\pm$ 17.03	76.28 $\pm$ 17.66	75.36 $\pm$ 17.36	76.17 $\pm$ 17.95	77.30 $\pm$ 17.14	+0.58 $\pm$ 1.31	+1.95 $\pm$ 2.04
ViT-B/32	62.11 $\pm$ 20.79	71.12 $\pm$ 18.44	72.84 $\pm$ 18.46	71.60 $\pm$ 18.43	72.83 $\pm$ 18.43	73.15 $\pm$ 18.53	+0.48 $\pm$ 1.40	+1.55 $\pm$ 1.80
RN50	58.81 $\pm$ 20.90	66.88 $\pm$ 18.03	69.85 $\pm$ 19.00	66.73 $\pm$ 18.39	69.42 $\pm$ 19.46	70.87 $\pm$ 18.66	$-0.15\pm 4.52$	+4.13 $\pm$ 3.20
RN101	59.59 $\pm$ 22.53	69.42 $\pm$ 18.40	72.25 $\pm$ 18.49	69.58 $\pm$ 18.35	71.72 $\pm$ 19.20	73.35 $\pm$ 18.03	+0.16 $\pm$ 2.62	+3.77 $\pm$ 2.81

Appendix

Table S21 : Configuration-transfer results at

K=4

, using the same notation and dispersion definitions as Tab. S20 .

$K$	Comparison	Mean $\pm$ sd	Median	Wins	Sign $p$	Wilcoxon $p$
16	$3-1$	+0.611 $\pm$ 2.771	+0.593	28/44	0.096	0.126
16	$4-2$	+0.941 $\pm$ 1.269	+0.561	36/44	$2.5\!\times\!10^{-5}$	$4.6\!\times\!10^{-8}$
16	$2-3\mathrm{a}$	+0.283 $\pm$ 0.875	+0.087	34/43	$1.7\!\times\!10^{-4}$	$2.1\!\times\!10^{-3}$
16	$3\mathrm{a}-3$	+1.046 $\pm$ 1.487	+0.338	30/42	$7.9\!\times\!10^{-3}$	$3.1\!\times\!10^{-5}$
16	$4-3$	+2.270 $\pm$ 2.276	+1.420	41/44	$1.6\!\times\!10^{-9}$	$1.2\!\times\!10^{-10}$
4	$3-1$	+0.267 $\pm$ 2.698	+0.512	28/44	0.096	0.310

Appendix

Table S22 : Paired comparisons between transfer configurations. Differences are computed after averaging the three runs within each of the 44 dataset–backbone settings. Sign-test and Wilcoxon

p

-values are two-sided.

Backbone	GDA-CLIP (final-feature)	LE-GDA	Tip-Adapter
RN50	1,025	3,074	16,400
RN101	513	2,562	8,208
ViT-B/32	513	2,051	8,208
ViT-B/16	513	2,051	8,208

Appendix

Table S24 : Median number of stored floating-point values per class at

K=16

. Tip-Adapter uses sparse integer labels in this accounting.

VLM Adaptation

Vision-language model adaptation tailors pretrained image-text and multimodal models to new tasks and domains while retaining transferable capabilities.

See all 545 papers

Oct 15, 2026

LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation

Few-shot adaptation often relies on an encoder's final global representation, leaving complementary information in intermediate layers underused. Combining these features is challenging because their dimensions can differ and their usefulness varies across tasks. We propose LAMP, Layerwise Adaptive Mixing of Predictors, a training-free framework that combines global and regional information from multiple encoder layers. A single forward pass per view supplies all descriptors. Global and support-calibrated regional kernel predictors independently fit the same label residuals. Labeled validation data select their mixture in class-score space, preserving native feature dimensions without learned projections. LAMP-F further adapts text-anchored class prototypes, using a cached response to update the fixed-kernel residual predictions exactly. Experiments cover eleven datasets and ViT and ResNet backbones. LAMP achieves state-of-the-art mean accuracy among the compared published training-free ResNet-50 methods at 4, 8, and 16 shots, reaching 78.73% at 16 shots. Averaged across five support budgets, LAMP improves over a paired global-feature-only baseline by 1.14 and 1.59 percentage points on ViT-B/16 and ResNet-50; LAMP-F raises these gains to 1.96 and 2.54 points. LAMP-F reaches 16-shot accuracies of 83.57% and 79.10%, respectively. Controlled experiments identify multi-layer integration as the main source of improvement and examine pooling, calibration, validation labels, and computational cost. These findings show that combining layer-specific predictions can exploit complementary features from frozen encoders to improve few-shot classification.

Zhipeng Ye, Feng Jiang, Qiufeng Wang +1

Taizhou Institute of Science and Technology, Nanjing University of Science and Technology, Taizhou 225300, Jiangsu, China · Department of Intelligence Science, Xi’an Jiaotong-Liverpool University, Suzhou 215123, Jiangsu, China · School of Computer Science and Technology, University of Arizona, Tucson 85705, AZ, USA

Few-Shot Learning Few-Shot Image Classification VLM Adaptation

Ask PDF

May 28, 2026

Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning

Vision-Language Models (VLMs) such as CLIP demonstrate strong zero-shot generalization, but their performance significantly degrades in cross-domain scenarios with scarce target-domain training data (Cross-Domain Few-Shot Learning, CDFSL). In this paper, we focus on the target-domain few-shot finetuning in the CLIP-based CDFSL task. Prevailing finetuning paradigms uniformly align all image patch tokens with their corresponding textual embeddings. However, we find a counterintuitive phenomenon: actively pushing away certain low-similarity image tokens, termed "tail tokens", from their textual embeddings consistently improves target-domain performance. We delve into this phenomenon and provide a novel interpretation: under great domain shifts and scarce training data, the model can hardly extract semantic information from visual inputs; therefore, the common belief of alignment is valid only for tokens already containing sufficient semantic information; for tail tokens, forcing the alignment would lead to excessive overfitting to the scarce training, while breaking the alignment is more useful. Motivated by this, we propose Adaptive Tail-Head Alignment (ATHA), a novel fine-tuning strategy for CLIP that transforms the conventional uniform alignment paradigm to an adaptive alignment paradigm, with both alignment strengthening and weakening. Extensive experiments on four challenging CDFSL benchmarks validate our state-of-the-art performance. Our code is available at https://github.com/shuaiyi308/ATHA.

Shuai Yi, Yixiong Zou, Yuhua Li +1

Institute of Artificial Intelligence, Huazhong University of Science and Technology, Wuhan, China · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China.

Source-Free Cross-Domain Few-Shot Learning Source-Free Domain Adaptation Few-Shot Domain Adaptation VLM Adaptation Few-Shot Learning Vision-Language Alignment

Ask PDF

Mar 4, 2026

Hold-One-Shot-Out (HOSO) for Validation-Free Few-Shot CLIP Adapters

In many CLIP adaptation methods, a blending ratio hyperparameter controls the trade-off between general pretrained CLIP knowledge and the limited, dataset-specific supervision from the few-shot cases. Most few-shot CLIP adaptation techniques report results by ablation of the blending ratio on the test set or require additional validation sets to select the blending ratio per dataset, and thus are not strictly few-shot. We present a simple, validation-free method for learning the blending ratio in CLIP adaptation. Hold-One-Shot-Out (HOSO) presents a novel approach for CLIP-Adapter-style methods to compete in the newly established validation-free setting. CLIP-Adapter with HOSO (HOSO-Adapter) learns the blending ratio using a one-shot, hold-out set, while the adapter trains on the remaining few-shot support examples. Under the validation-free few-shot protocol, HOSO-Adapter outperforms the CLIP-Adapter baseline by more than 4 percentage points on average across 11 standard few-shot datasets. Interestingly, in the 8- and 16-shot settings, HOSO-Adapter outperforms CLIP-Adapter even with the optimal blending ratio selected on the test set. Ablation studies validate the use of a one-shot hold-out mechanism, decoupled training, and improvements over the naively learnt blending ratio baseline. Code is released here: https://github.com/chris-vorster/HOSO-Adapter

Chris Vorster, Mayug Maniparambil, Noel E. O'Connor +2

ML-Labs, Dublin City University, Dublin, Ireland

VLM Adaptation Few-Shot Learning Few-Shot Image Classification

Ask PDF

Abstract

VLM Adaptation

Explore similar work

LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation

Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning

Hold-One-Shot-Out (HOSO) for Validation-Free Few-Shot CLIP Adapters