EGiME: Entropy-Guided Model Evaluation in Few-Shot Learning

Dequan Jin; Tao Li; Lifeng Xing; Shihui Ying; Hui Zhao

cs.LGOct 19, 2026

EGiME: Entropy-Guided Model Evaluation in Few-Shot Learning

Authors: Dequan Jin, Tao Li, Lifeng Xing, Shihui Ying, Hui Zhao

Organizations: Department of Mathematics and Applied Mathematics, School of Mathematics and Information Science, Guangxi University, Nanning, Guangxi, 530004, China · Department of Mathematics, School of Science, Shanghai University, Shanghai, 200444, China · School of Economics, Guangxi University, Nanning, Guangxi, 530004, China

Abstract

The feature-extraction quality of pre-trained models sets the ceiling of downstream transfer learning, yet few-shot model selection remains difficult: supervised evaluation on scarce labels is unstable and yields fluctuating rankings. We propose Entropy-Guided Model Evaluation (EGiME), a semi-supervised criterion that assesses a candidate model from the clustering structure of its feature space without any classifier training. EGiME performs $K$ -means clustering over all labeled and unlabeled samples, then combines two entropy terms: an inter-cluster entropy measuring global separability, and an intra-cluster entropy measuring the compactness of labeled anchors, fused through an inverse-frequency weight that emphasizes small clusters. On five few-shot benchmarks and nine pre-trained models, EGiME achieves a correlation score of 0.966 with the full-pool ground-truth accuracy obtained by averaging the accuracies of K-means, KNN, and SVM. At labeling ratios of 4%--20%, it produces markedly more stable model rankings than cross-validated supervised baselines, establishing it as a practical alternative to conventional supervised evaluation for pre-trained model selection under data scarcity.

Figures & tables

Panel of Figure 1 — Figure 1: The feature-extracted sample distributions derived by AlexNet (left) and ViT-B/16 (right) from the Mini-ImageNet dataset and visualized by $t$ -SNE. Different colors represent different categories.

Figure 2: Overall framework of the proposed EGiME methodology. The pipeline consists of three main stages: Feature embedding, structure discovery, and entropy-based evaluation.

Algorithm 1 Fold-Averaged EGiME
Input: Raw image dataset $\mathcal{D}$ , model set $\mathcal{M}=\{M_{1},\dots,M_{q}\}$ , cluster number $K=s$ , labeled sample indices and corresponding labels, number of stratified folds $F$
Output: Fold-averaged weighted entropy vector $\bar{\mathbf{E}}=(\bar{E}_{1},\dots,\bar{E}_{q})^{T}$
1: for each model $M_{r}\in\mathcal{M}$ do
2: Extract the full-pool feature set $D_{f}^{(r)}$ and partition the labeled samples into $F$ stratified folds;
3: for each fold $j=1,\dots,F$ do
4: Compute the class-mean centroids of the current labeled fold, and run K-means once from these centroids ( $n_{\mathrm{init}}=1$ ) on the complete pool $D_{f}^{(r)}$ ;

Table 1: EGiME: Fold-Averaged Entropy-Guided Model Evaluation

Model	Architecture	Feature Dim.
AlexNet	CNN	4096
VGG11	CNN	4096
VGG13	CNN	4096
GoogLeNet	CNN	1024
ResNet18	CNN	512
ResNet34	CNN	512

Table 2: Specifications of the Pre-trained Models Evaluated in the Experiments

Figure 3: EGiME as a low-label proxy for the three-predictor ground-truth accuracy. The overall regression score of

R^{2}=0.966

indicates that EGiME preserves the accuracy trend and can serve as a practical pre-screening alternative to fully supervised model selection.

Dataset	AlexNet	VGG11	VGG13	GoogLeNet	ResNet18	ResNet34	ViT-S	ViT-B	ViT-L
Mini-val	7	9	8	6	5	3	4	1	2
STL-10	7	8	9	6	5	4	3	2	1
CIFAR-10	7	8	9	4	5	3	6	1	2
EuroSAT	7	8	9	4	3	1	5	6	2
CIFAR-100	7	8	9	6	5	4	2	3	1

Table 3: Ground-Truth Rankings of Pre-trained Models on Five Benchmark Datasets, Obtained by Averaging the Accuracies of K-means, KNN, and SVM

Dataset	Method	Labels Per Class ( $n_{\mathrm{shot}}$ )
Dataset	Method	4	6	8	10	12	14	16	18	20
Mini-val	CV(KNN)	3.01	2.88	2.72	2.43	2.25	2.33	2.23	1.92	1.81
	CV(SVM)	3.38	3.74	4.56	4.70	4.82	4.59	4.70	4.64	4.53
	LEEP	12.59	11.91	12.04	12.22	11.82	11.80	11.83	11.54	11.28
	LogME	3.87	3.96	4.14	4.12	4.29	4.26	4.30	4.22	4.24
	PACTran	8.23	8.78	9.24	9.31	9.42	9.52	9.62	9.71	9.74

Table 4: Average Ranking Mistakes on Five Datasets Under Different Labeling Proportions

Panel of Figure 4 — Figure 4: Average ranking mistakes of the six methods as a function of the number of labeled samples per class, drawn as waterfall surfaces; the vertical axis is logarithmic, since LEEP and PACTran exceed the remaining methods by up to an order of magnitude. Within each panel the methods run from front (EGiME) to back (LEEP).

Panel of Figure 5 — Figure 5: Score distributions across different labeled sample sizes. (A) Mini-ImageNet, (B) STL-10, (C) CIFAR-10. Left: EGiME, Middle: CV(KNN), Right: CV(SVM).

Panel of Figure 6 — Figure 6: Score distributions across different labeled sample sizes (continued). (D) EuroSAT, (E) CIFAR-100.

Dataset	Strategy	Labels Per Class ( $n_{\mathrm{shot}}$ )
Dataset	Strategy	4	6	8	10	12	14	16	18	20
Mini-val	K-means	2.87	2.62	2.09	2.00	1.92	1.83	1.70	1.54	1.38
Mini-val	Equal Partition	35.97	35.97	35.97	35.97	35.97	35.97	35.97	35.97	35.97
STL-10	K-means	2.14	1.71	1.60	1.37	1.30	1.29	1.28	1.11	1.18
STL-10	Equal Partition	13.63	14.95	15.41	15.16	17.85	13.06	16.41	14.06	17.85
CIFAR-10	K-means	3.66	3.48	3.39	3.11	2.83	2.28	2.75	2.40	2.15

Table 5: Ablation Study (A): Ranking Errors of K-means vs. Equal Partition

Dataset	Average $\Delta RE$
Dataset	$E_{r}^{\mathrm{A}}=E_{r}^{\mathrm{(inter)}}$	$E_{r}^{\mathrm{B}}=1/(1+E_{r}^{\mathrm{(intra)}})$
Mini-val	+1.00	+2.00
STL-10	+1.33	+1.33
CIFAR-10	+3.89	+3.00
EuroSAT	+1.78	+10.11
CIFAR-100	+0.11	+4.11

Table 6: The contribution of

E_{r}^{\mathrm{A}}

and

E_{r}^{\mathrm{B}}

. Increase in Ranking Error (

\Delta RE=RE_{\mathrm{variant}}-RE_{\mathrm{Full}}

) after Removing Each Entropy Component. Smaller

\Delta RE

indicates the removed component is less critical.

Model Selection

Model selection chooses models, features, or configurations that balance predictive fit, generalization, complexity, and deployment constraints.

See all 96 papers

Benchmarking Few-shot Transferability of Pre-trained Models with Improved Evaluation Protocols
Feb 28, 2026Xu Luo, Ji Zhang, Lianli Gao +2Few-Shot Learning Transfer Learning
University of Electronic Science and Technology of China, Chengdu, China. · Southwest Jiaotong University, Chengdu, China. · Tongji University, Shanghai, China.
Are We Really Doing Few-Shot Learning? A Critical Examination of Pre-Training Assumptions
Sep 9, 2026Alejandro Galan-Cuenca, Marcelo Saval-Calvo, Antonio Javier GallegoDomain Adaptation Few-Shot Learning
University Institute for Computer Research, University of Alicante, Alicante, Spain
Geometry-Aware Adaptation for Pretrained Models
Jul 23, 2023Nicholas Roberts, Xintong Li, Dyah Adila +4Zero-Shot Learning VLM Adaptation
University of Wisconsin-Madison

Feb 28, 2026J/K move · Enter open · S save

Benchmarking Few-shot Transferability of Pre-trained Models with Improved Evaluation Protocols

Xu Luo, Ji Zhang, Lianli Gao, Heng Tao Shen+1

University of Electronic Science and Technology of China, Chengdu, China. · Southwest Jiaotong University, Chengdu, China. · Tongji University, Shanghai, China.

Few-Shot Learning Transfer Learning Fine-Tuning

Open paper AskPDF arXiv

Few-shot transfer has been revolutionized by stronger pre-trained models and improved adaptation algorithms.However, there lacks a unified, rigorous evaluation protocol that is both challenging and realistic for real-world usage. In this work, we establish FEWTRANS, a comprehensive benchmark containing 10 diverse datasets, and propose the Hyperparameter Ensemble (HPE) protocol to overcome the "validation set illusion" in data-scarce regimes. Our empirical findings demonstrate that the choice of pre-trained model is the dominant factor for performance, while many sophisticated transfer methods offer negligible practical advantages over a simple full-parameter fine-tuning baseline. To explain this surprising effectiveness, we provide an in-depth mechanistic analysis showing that full fine-tuning succeeds via distributed micro-adjustments and more flexible reshaping of high-level semantic presentations without suffering from overfitting. Additionally, we quantify the performance collapse of multimodal models in specialized domains as a result of linguistic rarity using adjusted Zipf frequency scores. By releasing FEWTRANS, we aim to provide a rigorous "ruler" to streamline reproducible advances in few-shot transfer learning research. We make the FEWTRANS benchmark publicly available at https://github.com/Frankluox/FewTrans.

Abstract

Model Selection

Explore similar work

Benchmarking Few-shot Transferability of Pre-trained Models with Improved Evaluation Protocols

Are We Really Doing Few-Shot Learning? A Critical Examination of Pre-Training Assumptions

Geometry-Aware Adaptation for Pretrained Models