Federated Active Learning Under Extreme Non-IID and Global Class Imbalance

Chen-Chen Zong; Sheng-Jun Huang

cs.LGMar 11, 2026

Federated Active Learning Under Extreme Non-IID and Global Class Imbalance

Authors: Chen-Chen Zong, Sheng-Jun Huang

Organizations: Nanjing University of Aeronautics and Astronautics, China

Abstract

Federated active learning (FAL) seeks to reduce annotation cost under privacy constraints, yet its effectiveness degrades in realistic settings with severe global class imbalance and highly heterogeneous clients. We conduct a systematic study of query-model selection in FAL and uncover a central insight: the model that achieves more class-balanced sampling, especially for minority classes, consistently leads to better final performance. Moreover, global-model querying is beneficial only when the global distribution is highly imbalanced and client data are relatively homogeneous; otherwise, the local model is preferable. Based on these findings, we propose FairFAL, an adaptive class-fair FAL framework. FairFAL (1) infers global imbalance and local-global divergence via lightweight prediction discrepancy, enabling adaptive selection between global and local query models; (2) performs prototype-guided pseudo-labeling using global features to promote class-aware querying; and (3) applies a two-stage uncertainty-diversity balanced sampling strategy with k-center refinement. Experiments on five benchmarks show that FairFAL consistently outperforms state-of-the-art approaches under challenging long-tailed and non-IID settings. The code is available at https://github.com/chenchenzong/FairFAL.

Figures & tables

Figure 1: Comparison of global (G) and local (L) query selectors under varying

(\alpha,\rho)

configurations on CIFAR-10. Test accuracy is plotted against the proportion of labeled samples for Entropy and Coreset sampling.

Figure 2: Cumulative sampling proportions of majority, intermediate, and minority classes for global (G) and local (L) query selectors under different

(\alpha,\rho)

settings on CIFAR-10, where the three groups denote the top 3, middle 4, and bottom 3 classes by sample proportion.

Panel of Table 1 — Table 1: Pairwise comparison between global (G) and local (L) query selectors under different $(\alpha,\rho)$ configurations on CIFAR-10. Each entry reports the winner (strategy $i$ ), positive ratio ( $\pi^{+}_{ij}$ ), one-sided Wilcoxon $p$ -value, and Hodges–Lehmann effect size $\widehat{\Delta}_{HL}^{ij}$ (in percentage points, pp) for Entropy and Coreset sampling.

Figure 3: Test accuracy vs. proportion of labeled samples on FMNIST, CIFAR-10, and CIFAR-100 under global imbalance

\rho=20

and two heterogeneity levels (

\alpha=0.1

and

\alpha=100

). Each subplot corresponds to a dataset–heterogeneity setting, and curves compare FairFAL with FAL and AL baselines using global (G-) or local (L-) models. Numerical results are provided in Tables 5 and 6 in the Appendix.

Panel of Table 2 — Table 2: Ablation results of FairFAL under different $(\alpha,\rho)$ configurations. We evaluate three key components: adaptive model selection, prototype-guided class-wise sampling, and the two-stage uncertainty–diversity balanced sampling strategy. Modules are added sequentially from left to right, and the final configuration corresponds to $\kappa{=}4$ . All results are reported in terms of final-round test accuracy (%).

	Random	KAFAL	LoGo	IFAL	Ours
OctMNIST	68.30	70.40	70.00	68.40	72.80
DermaMNIST	72.32	73.27	73.62	72.97	73.77

Table 3: Final-round test accuracy comparison on OctMNIST and DermaMNIST under extreme client heterogeneity

\alpha{=}0.1

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Visualization of the data distributions under the four $(\alpha,\rho)$ configurations discussed in the Observation section. These plots illustrate how client heterogeneity and global class imbalance jointly shape the federated data landscape.

Dataset	#Classes	Train / Test	Imbalance
FMNIST [ 53 ]	10	60k / 10k	Balanced
CIFAR-10 [ 29 ]	10	50k / 10k	Balanced
CIFAR-100 [ 29 ]	100	50k / 10k	Balanced
OCTMNIST [ 55 ]	4	97,477 / 10,832	Long-tailed
DermaMNIST [ 55 ]	7	7,007 / 2,003	Long-tailed

Appendix

Table 4: Summary of datasets used in our experiments, including natural-image (FMNIST, CIFAR-10, and CIFAR-100) and medical-image benchmarks (OCTMNIST and DermaMNIST). Long-tailed indicates naturally imbalanced label distributions.

Method	Model	FMNIST				CIFAR-10				CIFAR-100
Method	Model	15%	25%	35%	45%	15%	25%	35%	45%	15%	25%	35%	45%
Random	–	83.06	84.88	85.01	85.60	47.24	50.46	54.29	55.70	17.69	22.44	24.13	27.44
Entropy [ 19 ]	$G$	85.20	85.71	86.48	86.72	50.15	54.79	56.76	58.54	16.89	22.01	23.74	27.15
Entropy [ 19 ]	$L$	85.49	86.36	86.76	87.06	50.93	56.26	58.34	59.35	17.44	22.52	24.56	27.70
Margin [ 6 ]	$G$	85.21	85.71	86.23	86.53	49.66	53.90	56.89	58.24	17.52	22.46	24.47	27.70
Margin [ 6 ]	$L$	85.43	86.16	86.76	87.04	51.16	55.64	58.28	59.49	17.51	22.71	24.80	27.92

Appendix

Table 5: Test accuracy (%) comparison on three benchmark datasets under different labeled-data ratios, with

\alpha=0.1

and

\rho=20

. Results are averaged over five random seeds. Traditional AL baselines are evaluated using two query-selector models,

G

and

L

, denoting global and local selectors, respectively. Bold numbers indicate the best result in each column, while underlined numbers denote the second best.

Method	Model	FMNIST				CIFAR-10				CIFAR-100
Method	Model	15%	25%	35%	45%	15%	25%	35%	45%	15%	25%	35%	45%
Random	–	84.26	85.96	86.76	87.29	48.51	52.96	58.00	59.70	18.23	23.57	25.09	28.93
Entropy [ 19 ]	$G$	85.88	87.28	88.14	88.46	52.43	57.47	61.53	63.83	17.56	22.98	24.80	28.77
Entropy [ 19 ]	$L$	85.98	87.34	88.50	88.80	51.37	56.59	60.01	61.74	17.33	22.66	24.88	27.97
Margin [ 6 ]	$G$	85.84	86.92	88.18	88.38	51.26	57.93	62.13	63.72	18.68	23.92	25.76	28.91
Margin [ 6 ]	$L$	85.89	87.39	88.29	88.61	50.09	55.59	59.58	61.68	18.27	23.71	25.78	28.70

Appendix

Table 6: Test accuracy (%) comparison on three benchmark datasets under different labeled-data ratios, with

\alpha=100

and

\rho=20

. Results are averaged over five random seeds. Traditional AL baselines are evaluated using two query-selector models,

G

and

L

, denoting global and local selectors, respectively. Bold numbers indicate the best result in each column, while underlined numbers denote the second best.

Figure 5: Test accuracy versus labeled-data ratio on CIFAR-10 under global imbalance (

\rho{=}20

) and two heterogeneity levels (

\alpha{=}0.1

and

\alpha{=}100

), evaluated with FedProx and SCAFFOLD. Each subplot shows the corresponding FL framework and heterogeneity setting, comparing FairFAL with FAL and AL baselines using either the global (G-) or local (L-) query model.

Figure 6: Test accuracy versus labeled-data ratio on CIFAR-10 under global imbalance (

\rho{=}20

) and two heterogeneity levels (

\alpha{=}0.1

and

\alpha{=}100

), evaluated with either

5

20

clients. Each subplot corresponds to a specific combination of client count and heterogeneity level, and compares FairFAL with FAL and AL baselines using either the global (G-) or local (L-) query model.

Method	CIFAR-10
	$\alpha=0.1$ , $\rho=20$				$\alpha=100$ , $\rho=20$
	15%	25%	35%	45%	15%	25%	35%	45%
FairFAL (LC)	51.64	56.83	59.22	60.56	53.17	59.65	63.25	64.27
FairFAL (Margin)	51.78	56.82	58.94	60.48	52.82	59.63	63.05	64.57
FairFAL (Entropy)	52.12	56.90	59.62	60.44	53.25	59.17	62.62	64.57

Appendix

Table 7: Test accuracy versus labeled-data ratio of FairFAL under different uncertainty measures on CIFAR-10 with

\alpha{=}0.1

and

\alpha{=}100

(global imbalance

\rho{=}20

). We replace the default entropy-based uncertainty with Margin and Least Confidence (LC) scoring.

Method	CIFAR-10
	$\alpha=0.1$ , $\rho=20$				$\alpha=100$ , $\rho=20$
	15%	25%	35%	45%	15%	25%	35%	45%
FairFAL ( $\delta=0.65$ )	52.25	56.89	59.33	60.27	53.25	59.17	62.62	64.57
FairFAL ( $\delta=0.70$ )	52.12	56.90	59.62	60.44	53.25	59.17	62.62	64.57
FairFAL ( $\delta=0.75$ , Final )	52.12	56.90	59.62	60.44	53.25	59.17	62.62	64.57
FairFAL ( $\delta=0.80$ )	52.12	56.90	59.62	60.44	53.47	59.34	62.44	64.32

Appendix

Table 8: Test accuracy versus labeled-data ratio of FairFAL under different model-selection thresholds

\delta

on CIFAR-10 with

\alpha{=}0.1

and

\alpha{=}100

(global imbalance

\rho{=}20

). We evaluate

\delta\!\in\!\{0.65,0.70,0.75,0.80,0.85\}

, where

\delta{=}0.75

corresponds to the default configuration.

Method	MobileNet		ResNet-18
Method	$\alpha=0.1$	$\alpha=100$	$\alpha=0.1$	$\alpha=100$
KAFAL	41.16	43.14	37.96	42.57
LoGo	39.08	45.87	37.10	44.95
IFAL	36.64	41.80	36.89	42.10
FairFAL	41.07	46.66	38.42	45.53

Appendix

Table 9: Final-round test accuracy (seed = 1) of KAFAL, LoGo, IFAL, and FairFAL under different backbone architectures on CIFAR-10 with heterogeneity levels

\alpha=0.1

and

\alpha=100

(global imbalance

\rho=20

). The default 4-layer CNN backbone is replaced with MobileNet and ResNet-18, respectively.

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 288 papers

Coordination on a Budget: Federated Active Learning with Few Labels
Aug 19, 2026Liam Mohr, Daphna WeinshallActive Learning Privacy-Preserving ML
School of Computer Science and Engineering The Hebrew University of Jerusalem, Jerusalem 91904, Israel
FIDAL: Diversity-Aware Federated Active Learning Under Real-World Distribution Shifts
Sep 10, 2026David Dueñas Gaviria, Shadi AlbarqouniActive Learning Federated Learning
University of Bonn, University Hospital Bonn, Clinic for Diagnostic and Interventional Radiology, Venusberg-Campus 1, 53127 Bonn, Germany
FedReLa: Imbalanced Federated Learning via Re-Labeling
Jun 24, 2026Guangzheng Hu, Patricia Menéndez, Feng Liu +3Class-Imbalanced Learning Federated Learning
School of Mathematics and Statistics, University of Melbourne, Victoria, Australia · School of Computing and Information Systems, University of Melbourne, Victoria, Australia · School of Statistics and Data Science, LPMC, KLMDASR, and LEBPS, Nankai University

Aug 19, 2026J/K move · Enter open · S save

Coordination on a Budget: Federated Active Learning with Few Labels

Liam Mohr, Daphna Weinshall

School of Computer Science and Engineering The Hebrew University of Jerusalem, Jerusalem 91904, Israel

Active Learning Privacy-Preserving ML Federated Learning

Open paper AskPDF arXiv

Federated Active Learning (FAL) addresses the dual challenges of data privacy and label scarcity, where the absence of a global data view introduces additional hurdles for coordinated query selection. We study cross-silo FAL in the low-budget regime, where annotation decisions are most critical. We characterize, both theoretically and empirically, a heterogeneity reversal: in low-budget settings, homogeneous (IID) data requires stronger coordination to avoid redundant queries, whereas heterogeneous data naturally promotes diversity; this trend reverses at higher budgets. Thus, in contrast to the standard federated learning (FL) narrative where heterogeneity is a primary challenge, we show that IID settings are more challenging for query selection in FAL. Motivated by these findings, we propose a new FAL framework that utilizes federated representation learning to align client data in a shared embedding space. This enables the server to perform globally coordinated active selection over optionally obfuscated client embeddings, while annotation remains local to each client. Although our framework operates in the more challenging low-budget regime, it achieves performance that surpasses existing FAL methods even when they are given substantially larger annotation budgets, demonstrating the value of centralized coordination under privacy constraints.

Abstract

Active Learning

Explore similar work

Coordination on a Budget: Federated Active Learning with Few Labels

FIDAL: Diversity-Aware Federated Active Learning Under Real-World Distribution Shifts

FedReLa: Imbalanced Federated Learning via Re-Labeling