Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data

Morris Stallmann; Charalampos S. Kouzinopoulos; Marcin Pietrasik; Anna Wilbik

cs.LGSep 18, 2026

Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data

Authors: Morris Stallmann, Charalampos S. Kouzinopoulos, Marcin Pietrasik, Anna Wilbik

Organizations: Department of Advanced Computing Sciences (DACS) Maastricht University Maastricht, Netherlands

Abstract

Clustering high-dimensional data is a fundamental task in unsupervised machine learning with applications to a variety of domains. In the centralized data scenario, this task is commonly solved using deep clustering methods that utilize deep neural network architectures to learn clustering-friendly latent space representations. In Federated Learning, where data is distributed between clients and is private, deep clustering methods are less explored. In particular, recently introduced federated deep clustering methods, despite showing very promising performance, still fall short in reliably providing good performance if data across clients are non-identically-independently distributed. In this work, we introduce a generalization of Deep Clustering Networks to the federated scenario, named FedDCN, that simultaneously optimizes a reconstruction loss and a clustering loss. To ensure robustness and latent space alignment in non-identically-independently distributed data scenarios, FedDCN generates synthetic data augmentations, and its learning objective includes a geometric regularization for latent space alignment. Through experimental evaluation, the effectiveness of the approach under IID and non-IID assumptions is demonstrated, and future research directions are identified.

Figures & tables

Fig. 1: Illustration of the three stages in FedDCN training: synthetic data creation, pretraining, federated training.

Hyperparameter	Search Space
AE layer dimensions	{ (128), (256, 128), (512, 256), (512, 256, 128), (1024, 512, 256, 128, 64) }
Bottleneck dimension $d$	$[3,16]$
$\beta^{pre}$ during pretraining	$[0.0001,1.0]$
$\beta^{fed}$ during federated training	$[0.0001,1.0]$
$\lambda$	$[0.0001,1.0]$
$\alpha$	$[0.1,10.0]$

TABLE I: Hyperparameters search space.

MNIST
Clients	Scenario	Accuracy	NMI	DBI (latent space)
5	IID	0.8026 (0.7643 $\pm 0.0525$ )	0.7592 (0.7445 $\pm 0.0144$ )	0.6915 (0.7395 $\pm 0.0391$ )
5	Non-IID	0.7799 (0.7328 $\pm 0.0515$ )	0.7922 (0.7693 $\pm 0.0221$ )	0.9905 (1.0758 $\pm 0.0826$ )
20	IID	0.7160 (0.6838 $\pm 0.0206$ )	0.6772 (0.6666 $\pm 0.0103$ )	0.7462 (0.7929 $\pm 0.0396$ )
20	Non-IID	0.7636 (0.7029 $\pm 0.0475$ )	0.7442 (0.6981 $\pm 0.0423$ )	1.066 (1.1555 $\pm 0.0925$ )
Fashion-MNIST

TABLE II: Best results on the MNIST, Fashion-MNIST, and USPS datasets (avg. over five runs using the same hyperparameters in parantheses). *Results are obtained using a different latent space sampling strategy during synthetic data creation due to the small dataset size.

MNIST IID
		F-DEC	FDEC	FedDCN
	ACC	-	0.74	0.72
	NMI	0.65	0.68	0.68
MNIST Non-IID
	ACC	0.57	0.63	0.76
	NMI	0.48	0.58	0.74

TABLE III: Performance comparison of FedDCN (ours) and the results as reported by F-DEC [ 13 ] and FDEC [ 14 ] in federated IID and non-IID scenarios with 20 clients.

Fig. 2: Effect of removing the geometric loss on the NMI.

Data Scenario	AE layers	$d$	$\beta^{pre}$	$\beta^{fed}$	$\lambda$	$\alpha$	Learn. rate
MNIST IID	[256, 128]	3	0.0037	0.0204	0.0002	0.8273	0.0086
MNIST non-IID	[512, 256, 128]	4	0.0076	0.0003	0.0003	2.8550	0.0014
Fashion IID	[512, 256, 128]	6	0.0131	0.0003	0.2814	0.2137	0.0008
Fashion non-IID	[512, 256]	6	0.0304	0.0008	0.0005	0.3225	0.0001
USPS IID	[1024, 512, 256, 128, 64]	4	0.3711	0.0241	0.0470	0.1523	0.0002
USPS non-IID	[512, 256]	5	0.1970	0.0017	0.0006	0.5828	0.0024

TABLE IV: Optimal hyperparameters (optimized for accuracy).

Explore similar work

Fed-BRDECS: Privacy-Preserving and Heterogeneity-Aware Federated Deep Embedded Clustering
Oct 5, 2026Haemin Park, Diego Klabjan, Martin W. Braun +2Clustering Deep Clustering
Northwestern University · Intel Corporation
Discovering Collaboration from Novelty: Random Network Distillation for Clustered Federated Learning
Jun 29, 2026Davide Domini, Gianluca Aguzzi, Ivana Dusparic +2Unsupervised Clustering Non-IID Federated Learning
University of Bologna Cesena, Italy · Trinity College Dublin Dublin, Ireland
FMCL: Class-Aware Client Clustering with Foundation Model Representations for Heterogeneous Federated Learning
Apr 30, 2026Mahad Ali, Laura J. BrattainNon-IID Federated Learning Clustered Federated Learning
University of Central Florida, Orlando FL 32816, USA

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

Fed-BRDECS: Privacy-Preserving and Heterogeneity-Aware Federated Deep Embedded Clustering

Haemin Park, Diego Klabjan, Martin W. Braun, Xiuqi Li+1

Northwestern University · Intel Corporation

Clustering Deep Clustering Non-IID Federated Learning Privacy-Preserving ML Federated Learning

Open paper AskPDF arXiv

Federated deep clustering seeks to learn clustering-friendly representations from decentralized unlabeled data while preserving client privacy. However, Deep Embedded Clustering (DEC)-style objectives depend on global soft-assignment statistics that require clients to reveal their sensitive information. We propose Fed-BRDECS, a privacy-preserving and heterogeneity-aware federated deep embedded clustering framework. Fed-BRDECS replaces the globally normalized clustering objective with a locally computable sample-stability loss, avoiding the transmission of local soft-assignment distributions. To tackle non-IID client distributions, we introduce prediction-balanced sampling, which oversamples locally rare predicted clusters without requiring ground-truth labels, and centroid-level restarting, which periodically refreshes biased or inactive centroids. Experiments on image and text clustering benchmarks show that Fed-BRDECS consistently outperforms representative federated clustering and deep clustering baselines under both IID and non-IID partitions. We further demonstrate its applicability to federated time-series anomaly detection, where it improves reconstruction-based detectors without adding inference-time cost.

Abstract

Explore similar work

Fed-BRDECS: Privacy-Preserving and Heterogeneity-Aware Federated Deep Embedded Clustering

Discovering Collaboration from Novelty: Random Network Distillation for Clustered Federated Learning

FMCL: Class-Aware Client Clustering with Foundation Model Representations for Heterogeneous Federated Learning