Client-Conditional Federated Learning via Local Training Data Statistics

Rickard Brännvall

cs.LGMar 11, 2026

Client-Conditional Federated Learning via Local Training Data Statistics

Authors: Rickard Brännvall

Organizations: RISE Research Institutes of Sweden

Abstract

Federated learning (FL) under data heterogeneity remains challenging: existing methods either ignore client differences (FedAvg), require costly cluster discovery (IFCA), or maintain per-client models (Ditto). All degrade when data is sparse or heterogeneity is multi-dimensional. We propose conditioning a single global model on locally-computed PCA statistics of each client's training data, requiring zero additional communication. Evaluating across 97~configurations spanning four heterogeneity types (label shift, covariate shift, concept shift, and combined heterogeneity), four datasets (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100), and seven FL baseline methods, we find that our method matches the Oracle baseline -- which knows true cluster assignments -- across all settings, surpasses it by 1--6% on combined heterogeneity where continuous statistics are richer than discrete cluster identifiers, and is uniquely sparsity-robust among all tested methods.

Figures & tables

Exp.	Config	FedAvg	Oracle	Cond	Best Other
E1	CIFAR-10 $K{=}5$	.325	.927	.929	.906 (IFCA)
E2a	CIFAR-10 $K{=}3$	.943	.979	.981	.972 (DAC)
E2b	FMNIST rot. $K{=}4$	.789	.900	.895	.868 (DAC)
E3a	FMNIST sem. Rich	.671	.950	.950	.943 (IFCA)
E3b	CIFAR-10 perm. $K{=}2$	.408	.795	.808	.773 (DAC)
E4a	MNIST+FM Rich	.939	.957	.955	.953 (IFCA)

TABLE I : Summary of representative results across all experiments E1–E4. Bold indicates best non-Oracle method. Conditional consistently matches or exceeds Oracle.

Panel of Figure 2 — Fig. 1 : Client-conditional pipeline for client $i$ . Green boxes are client-local; orange boxes involve the federation. Prepare : PCA eigenvalues $\mathbf{s}_{i}$ are computed once from the training data. Train : model updates are computed locally and aggregated via federated learning to produce the shared model $\boldsymbol{\theta}$ . Infer : the client uses the shared model $\boldsymbol{\theta}$ and its own $\mathbf{s}_{i}$ for predictions.

Panel of Figure 3 — Fig. 2 : Four FL paradigms under data heterogeneity. (a) FedAvg: one global model, no personalization. (b) Clustered: $K$ separate models with iterative cluster discovery. (c) Personalized: $N$ per-client models fine-tuned from a shared model. (d) Ours: a single shared model conditioned on locally-computed statistics $\mathbf{s}_{i}$ , requiring no cluster discovery and no additional communication.

	MNIST				Fashion-MNIST				CIFAR-10
Method	$K{=}2$	$K{=}3$	$K{=}5$	$K{=}10$	$K{=}2$	$K{=}3$	$K{=}5$	$K{=}10$	$K{=}2$	$K{=}3$	$K{=}5$	$K{=}10$
Local	.991	.992	.997	1.00	.908	.920	.945	1.00	.767	.784	.856	1.00
FedAvg	.986	.978	.933	.522	.902	.886	.745	.412	.735	.554	.325	.172
Gossip	.982	.983	.892	.560	.895	.899	.825	.560	.745	.755	.533	.566
Ditto	.993	.994	.996	1.00	.910	.915	.933	.999	.826	.822	.843	1.00
IFCA	.995	.995	.999	1.00	.922	.932	.945	1.00	.847	.766	.906	1.00

TABLE II : E1: Label shift accuracy. Conditional matches Oracle across all dataset–

K

combinations. FedAvg collapses at high

K

while Conditional maintains Oracle-level performance.

Method	$C{=}2$ (4 cl.)	$C{=}3$ (6 cl.)	$C{=}4$ (8 cl.)	Mean
Local	.818	.854	.886	.853
FedAvg	.754	.743	.764	.754
Gossip	.775	.789	.786	.783
Ditto	.868	.880	.906	.885
IFCA	.752	.740	.892	.795
DAC	.888	.854	.896	.879

TABLE III : E4b: Combined heterogeneity on CIFAR-10. Conditional consistently exceeds Oracle because continuous statistics capture multi-dimensional variation that a discrete cluster ID cannot.

C

= covariate clusters per concept group; total clusters =

2C

Fig. 3 : Conditional exceeds Oracle on complex heterogeneity. Left: E3b label permutation on CIFAR-10 (

K

sweep). Right: E4b combined heterogeneity (concept + covariate shift). Continuous statistics provide richer conditioning than discrete cluster IDs.

	MNIST				Fashion-MNIST				CIFAR-10
Method	$K{=}2$	$K{=}3$	$K{=}4$	$K{=}5$	$K{=}2$	$K{=}3$	$K{=}4$	$K{=}5$	$K{=}2$	$K{=}3$	$K{=}4$	$K{=}5$
Local	.981	.977	.973	.969	.877	.863	.854	.845	.668	.618	.579	.558
FedAvg	.495	.332	.295	.306	.454	.330	.278	.257	.408	.304	.285	.282
Gossip	.691	.798	.568	.585	.619	.677	.477	.484	.545	.583	.429	.401
Ditto	.982	.980	.975	.970	.877	.854	.833	.815	.767	.725	.685	.661
IFCA	.990	.985	.982	.844	.892	.593	.659	.705	.411	.535	.717	.574

TABLE IV : E3b: Label permutation (

K

sweep). Conditional exceeds Oracle on all three datasets, with the largest gains on CIFAR-10.

Fig. 4 : Sparsity robustness on E1 Label Shift (

K{=}2

). Conditional and Oracle maintain flat accuracy as data decreases 20-fold (from

{\sim}6{,}000

{\sim}200

samples/client). All other methods degrade, with Gossip collapsing to near-random.

	Rich	Med.	Sparse	V. Sparse	S. Sparse	$\Delta$
	(4000)	(2000)	(800)	(400)	(200)
Local	.767	.713	.606	.526	.446	$-$ 41.9%
FedAvg	.709	.577	.421	.472	.434	$-$ 38.8%
Gossip	.752	.694	.532	.431	.211	$-$ 72.0%
Ditto	.825	.780	.695	.575	.496	$-$ 39.9%
IFCA	.852	.824	.765	.715	.631	$-$ 25.9%

TABLE V : E1 sparsity sweep on CIFAR-10 (

K{=}2

). Conditional maintains near-constant accuracy across a 20

\times

reduction in data per client.

\Delta

: relative degradation from Rich to Super Sparse.

Appendix figures & tables32 assets

Supplementary material from the paper’s appendix.

Appendix

	Reference models			Conditional models
Task	Global	Cluster	Client	Ens	MLP	Lin
Linreg (RMSE $\downarrow$ )	14.901	0.100	0.106	0.104	0.134	0.127
Logreg (acc $\uparrow$ )	0.700	0.997	0.944	0.989	0.985	0.964

Appendix

TABLE VI : MCDC synthetic tasks: 3 clusters, 100 clients each, 10 features. Lower RMSE is better for linear regression; higher accuracy is better for logistic regression. Best reference model and best conditional model in bold.

	Reference models			Conditional models
Task	Global	Cluster	Client	Ens	MLP	Lin
Linreg (RMSE $\downarrow$ )	17.655	0.100	0.107	0.109	0.162	0.257
Logreg (acc $\uparrow$ )	0.621	0.997	0.943	0.989	0.966	0.939

Appendix

TABLE VII : MCDC synthetic tasks: 8 clusters, 100 clients each, 10 features.

$n_{c}$	Global	Cluster	Client	Cond
0	0.844	0.969	0.884	—
1	0.847	0.970	0.880	0.967
2	0.848	0.971	0.878	0.967
3	0.847	0.971	0.876	0.966
4	0.848	0.971	0.874	0.966
8	0.850	0.972	0.871	0.966

Appendix

TABLE VIII : MCDC EMNIST: effect of number of PCA components (

n_{c}

) used for conditioning. A single component suffices; additional components provide no further benefit.

Fig. 5 : MCDC EMNIST: per-character accuracy for digits, uppercase, and lowercase character subsets (1 PCA component). The global model struggles with similar looking characters that appear in multiple subsets (e.g., lowercase ‘o’ vs uppercase ‘O’ vs digit ‘0’), while the conditional model resolves these ambiguities internally without having to identify peer clusters.

Group	Char	Global	Client	Cluster	Cond
o/O/0	o (lower)	.000	.578	.933	.917
	O (upper)	.517	.927	.978	.975
	0 (digit)	.824	.965	.993	.994
l/I/1	l (lower)	.227	.952	.965	.959
	I (upper)	.490	.895	.967	.968
	1 (digit)	.882	.988	.996	.996

Appendix

TABLE IX : MCDC EMNIST: accuracy on visually confusable characters across subsets. Characters are grouped by visual similarity; the global model fails on lowercase characters with cross-subset overlap, while the conditional model resolves these using client characteristic statistics only.

Fig. 6 : MCDC EMNIST: accuracy on confusable character pairs across subsets. The global model (rightmost) shows systematic failure on lowercase characters with cross-subset visual ambiguity, while the conditional model matches the cluster oracle.

	MNIST				FMNIST				CIFAR-10
Method	K=2	K=3	K=5	K=10	K=2	K=3	K=5	K=10	K=2	K=3	K=5	K=10
Local	.991	.992	.997	1.00	.908	.920	.945	1.00	.767	.784	.856	1.00
FedAvg	.986	.978	.933	.522	.902	.886	.745	.412	.735	.554	.325	.172
Gossip	.982	.983	.892	.560	.895	.899	.825	.560	.745	.755	.533	.566
Ditto	.993	.994	.996	1.00	.910	.915	.933	.999	.826	.822	.843	1.00
IFCA	.995	.995	.999	1.00	.922	.932	.945	1.00	.847	.766	.906	1.00

Appendix

TABLE X : E1: Label Shift Accuracy vs Number of Clusters (K)

Fig. 7 : E1: Test accuracy vs. number of clusters (K) for label shift heterogeneity across four datasets. Conditional (our method) matches Oracle performance across all settings.

	MNIST					FMNIST					CIFAR-10
Method	Rich	Med	Sparse	V.Sp	S.Sp	Rich	Med	Sparse	V.Sp	S.Sp	Rich	Med	Sparse	V.Sp	S.Sp
Local	.991	.986	.976	.963	.940	.909	.893	.866	.839	.798	.767	.713	.606	.526	.446
FedAvg	.986	.980	.950	.824	.738	.902	.884	.837	.784	.718	.709	.577	.421	.472	.434
Gossip	.982	.967	.873	.698	.151	.895	.855	.752	.633	.083	.752	.694	.532	.431	.211
Ditto	.993	.990	.978	.948	.916	.910	.891	.851	.786	.743	.825	.780	.695	.575	.496
IFCA	.994	.992	.985	.964	.935	.925	.906	.872	.824	.771	.852	.824	.765	.715	.631

Appendix

TABLE XI : E1: Label Shift Accuracy vs Sparsity (K=2, clients per cluster: 5/10/20/50/100)

Fig. 8 : E1: Method performance vs. data sparsity (K=2) for label shift. Conditional maintains Oracle-level accuracy across all sparsity levels.

Method	Accuracy (K=20)
Local	.543
FedAvg	.143
Gossip	.193
Ditto	.566
IFCA	.635
DAC	.651

Appendix

TABLE XII : E1: CIFAR-100 Label Shift Accuracy (K=20 superclasses)

Fig. 9 : E1: Complete results heatmap across all datasets and K values. Conditional (our method, bold) consistently matches Oracle performance.

	MNIST			FMNIST			CIFAR-10
Method	K=2	K=3	K=4	K=2	K=3	K=4	K=2	K=3	K=4
Local	.995	.996	.997	.996	.996	.998	.944	.955	.948
FedAvg	.989	.981	.981	.992	.986	.980	.956	.943	.923
Gossip	.988	.985	.980	.989	.988	.989	.946	.952	.932
Ditto	.996	.996	.996	.995	.993	.997	.960	.969	.961
IFCA	.997	.996	.989	.992	.996	.999	.954	.943	.961

Appendix

TABLE XIII : E2a: Covariate Shift (Subsampling) Accuracy vs K

Fig. 10 : E2a: Accuracy vs. K for covariate shift (subsampling). With local test evaluation, Conditional matches Oracle across all datasets.

	MNIST					FMNIST					CIFAR-10
Method	Rich	Med	Sparse	V.Sp	S.Sp	Rich	Med	Sparse	V.Sp	S.Sp	Rich	Med	Sparse	V.Sp	S.Sp
Local	.995	.992	.987	.980	.967	.995	.994	.992	.990	.989	.946	.926	.906	.881	.863
FedAvg	.988	.986	.977	.932	.847	.992	.988	.986	.985	.984	.956	.945	.932	.924	.916
Gossip	.988	.981	.897	.505	.504	.989	.988	.987	.600	.600	.951	.933	.907	.854	.833
Ditto	.997	.995	.989	.968	.928	.995	.993	.991	.990	.988	.961	.946	.936	.915	.892
IFCA	.998	.996	.991	.981	.958	.992	.989	.986	.986	.984	.954	.948	.933	.922	.915

Appendix

TABLE XIV : E2a: Covariate Shift Sparsity Sweep (K=2, clients per cluster: 5/10/20/50/100)

Fig. 11 : E2a: Accuracy vs. data sparsity (K=2) for covariate shift. Conditional is perfectly sparsity-invariant, while Gossip collapses at sparse settings.

	MNIST				FMNIST
Method	C1	C2	C3	C4	C1	C2	C3	C4
Local	.981	.976	.972	.971	.877	.860	.852	.851
FedAvg	.976	.962	.940	.965	.876	.836	.789	.821
Gossip	.968	.965	.931	.949	.859	.810	.718	.773
Ditto	.985	.976	.971	.979	.880	.846	.823	.837
IFCA	.990	.981	.977	.983	.880	.872	.860	.855

Appendix

TABLE XV : E2b: Rotation-Based Covariate Shift across four configurations

Fig. 12 : E2b: Rotation-based covariate shift results across all four configurations. Conditional closely tracks Oracle across all rotation settings.

	MNIST					FMNIST					CIFAR-10
Method	Rich	Med	Sparse	V.Sp	S.Sp	Rich	Med	Sparse	V.Sp	S.Sp	Rich	Med	Sparse	V.Sp	S.Sp
Local	.985	.979	.966	.948	.915	.930	.919	.901	.880	.854	.829	.774	.741	.707	.676
FedAvg	.793	.791	.785	.768	.713	.671	.661	.639	.628	.636	.762	.746	.742	.733	.727
Gossip	.871	.866	.506	.502	.500	.774	.746	.535	.500	.500	.804	.771	.716	.693	.649
Ditto	.988	.983	.963	.910	.765	.928	.918	.868	.815	.697	.878	.837	.804	.756	.718
IFCA	.992	.988	.962	.767	.714	.943	.934	.905	.876	.798	.759	.747	.740	.734	.726

Appendix

TABLE XVI : E3a: Semantic Concept Shift Accuracy (Sparsity Sweep, K=2, clients per cluster: 5/10/20/50/100)

Fig. 13 : E3a: Semantic concept shift sparsity sweep. Conditional maintains near-Oracle performance while other methods degrade significantly.

Fig. 14 : E3a: Gap to Oracle across sparsity levels (personalized methods only). Conditional stays closest to Oracle (negative gap = better than Oracle).

	MNIST				FMNIST				CIFAR-10
Method	K=2	K=3	K=4	K=5	K=2	K=3	K=4	K=5	K=2	K=3	K=4	K=5
Local	.981	.977	.973	.969	.877	.863	.854	.845	.668	.618	.579	.558
FedAvg	.495	.332	.295	.306	.454	.330	.278	.257	.408	.304	.285	.282
Gossip	.691	.798	.568	.585	.619	.677	.477	.484	.545	.583	.429	.401
Ditto	.982	.980	.975	.970	.877	.854	.833	.815	.767	.725	.685	.661
IFCA	.990	.985	.982	.844	.892	.593	.659	.705	.411	.535	.717	.574

Appendix

TABLE XVII : E3b: Label Permutation — K Sweep

Fig. 15 : E3b: Label permutation K sweep. Conditional consistently outperforms Oracle on CIFAR-10, demonstrating the value of continuous statistics over discrete cluster assignments.

	MNIST					FMNIST					CIFAR-10
Method	Rich	Med	Sparse	V.Sp	S.Sp	Rich	Med	Sparse	V.Sp	S.Sp	Rich	Med	Sparse	V.Sp	S.Sp
Local	.981	.973	.954	.929	.880	.878	.854	.814	.768	.715	.668	.576	.467	.406	.349
FedAvg	.494	.492	.481	.461	.399	.456	.440	.419	.388	.367	.411	.391	.354	.322	.286
Gossip	.691	.683	.107	.107	.106	.621	.584	.493	.100	.100	.544	.497	.425	.307	.207
Ditto	.983	.976	.949	.897	.766	.877	.848	.781	.737	.663	.769	.719	.612	.494	.394
IFCA	.990	.986	.970	.936	.869	.892	.873	.816	.774	.721	.412	.739	.675	.591	.502

Appendix

TABLE XVIII : E3b: Label Permutation — Sparsity Sweep (K=2, clients per cluster: 5/10/20/50/100)

Fig. 16 : E3b: Label permutation sparsity sweep (K=2). Gossip collapses to random at sparse settings; Conditional remains the best practical method.

Method	Rich	Medium	Sparse	Very Sparse
	(10 clients)	(20 clients)	(50 clients)	(100 clients)
Local	.942	.928	.908	.885
FedAvg	.939	.928	.904	.865
Gossip	.936	.920	.861	.109
Ditto	.944	.933	.910	.872
IFCA	.953	.944	.926	.896

Appendix

TABLE XIX : E4a: MNIST + FMNIST Domain Shift (Sparsity Sweep)

Fig. 17 : E4a: MNIST + FMNIST domain shift sparsity sweep. Conditional matches Oracle; Gossip collapses at Very Sparse.

Method	C=2 (4 clusters)	C=3 (6 clusters)	C=4 (8 clusters)
Local	.818	.854	.886
FedAvg	.754	.743	.764
Gossip	.775	.789	.786
Ditto	.868	.880	.906
IFCA	.752	.740	.892
DAC	.888	.854	.896

Appendix

TABLE XX : E4b: Combined Heterogeneity — Configuration Sweep (CIFAR-10)

Fig. 18 : E4b: Combined heterogeneity configuration sweep. Conditional exceeds Oracle across all configurations.

Method	Rich	Medium	Sparse	Very Sparse
	(20 clients)	(40 clients)	(100 clients)	(200 clients)
Local	.825	.793	.745	.706
FedAvg	.753	.742	.732	.727
Gossip	.770	.741	.686	.647
Ditto	.869	.833	.771	.737
IFCA	.753	.747	.734	.725

Appendix

TABLE XXI : E4b: Combined Heterogeneity — Sparsity Sweep (C=2, CIFAR-10)

Fig. 19 : E4b: Combined heterogeneity sparsity sweep (C=2). Conditional beats Oracle at Rich/Medium and remains competitive at sparse settings.

Experiment	Conditional vs Oracle	Key Observation
E1: Label Shift	Matches	Sparsity-invariant
E2a: Covariate (Subsampling)	Matches	Robust across K
E2b: Covariate (Rotation)	Matches	Works on transformed data
E3a: Semantic Concept Shift	Matches	Handles concept ambiguity
E3b: Label Permutation	Beats (+1–6%)	Statistics encode more than cluster ID
E4a: Cross-Dataset Domain	Matches	Clear domain separation

Appendix

TABLE XXII : Conditional vs Oracle Performance Across All Experiments

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 943 papers

Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning
May 16, 2025Shudi Weng, Chao Ren, Ming Xiao +1Federated Learning Heterogeneous Federated Learning
Division of Information Science and Engineering, School of Electrical Engineering and Computer Science (EECS), KTH Royal Institute of Technology, 11428 Stockholm, Sweden
Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation
Sep 15, 2026Polycarpo Souza Neto, José Mairton Barros da Silva Júnior, Charles Casimiro CavalcanteFederated Knowledge Distillation Non-IID Federated Learning
Universidade Federal do Cear´a, Fortaleza, CE, Brazil · Uppsala University, Uppsala, Sweden
FMCL: Class-Aware Client Clustering with Foundation Model Representations for Heterogeneous Federated Learning
Apr 30, 2026Mahad Ali, Laura J. BrattainNon-IID Federated Learning Clustered Federated Learning
University of Central Florida, Orlando FL 32816, USA

May 16, 2025J/K move · Enter open · S save

Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning

Shudi Weng, Chao Ren, Ming Xiao, Mikael Skoglund

Division of Information Science and Engineering, School of Electrical Engineering and Computer Science (EECS), KTH Royal Institute of Technology, 11428 Stockholm, Sweden

Federated Learning Heterogeneous Federated Learning Client Selection in Federated Learning

Open paper AskPDF arXiv

Federated learning (FL) commonly involves clients with diverse communication and computational capabilities. Such heterogeneity can significantly distort the optimization dynamics and lead to objective inconsistency, where the global model converges to an incorrect stationary point potentially far from the pursued optimum. Despite its critical impact, the joint effect of communication and computation heterogeneity has remained largely unexplored, due to the intrinsic complexity of their interaction. In this paper, we reveal the fundamentally distinct mechanisms through which heterogeneous communication and computation drive inconsistency in FL. To the best of our knowledge, this is the first unified theoretical analysis of general heterogeneous FL, offering a principled understanding of how these two forms of heterogeneity jointly distort the optimization trajectory under arbitrary choices of local solvers. Motivated by these insights, we propose Federated Heterogeneity-Aware Client Sampling, FedACS, a universal method to eliminate all types of objective inconsistency. We theoretically prove that FedACS converges to the correct optimum at a rate of

O(1/\sqrt{R})

, even in dynamic heterogeneous environments. Extensive experiments across multiple datasets show that FedACS outperforms state-of-the-art and category-specific baselines by 4.3%-36%, while reducing communication costs by 22%-89% and computation loads by 14%-105%, respectively.

Abstract

Federated Learning

Explore similar work

Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning

Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation

FMCL: Class-Aware Client Clustering with Foundation Model Representations for Heterogeneous Federated Learning