Evolving Federated Density Peak Clustering with Shuffler-Based Anonymization
Authors: Miha Ožbot, Seiichi Ozawa, Igor Škrjanc
Organizations: Faculty of Electrical Engineering, University of Ljubljana, Tržaška cesta 25, Ljubljana, 1000, Slovenia. · Center for Mathematical and Data Sciences, Kobe University, 1-1 Rokkodai-cho, Nada-ku, Kobe, 657-8501, Japan.
In this paper, we present a novel one-shot federated clustering approach based on evolving fuzzy clustering and shuffler-based anonymization. In federated clustering, each round reveals additional model information to the server and can increase the opportunity to infer properties of local datasets, even when raw samples are never shared. Consequently, fewer client-server exchanges and unlinkability are important, so transmitted information cannot be attributed to any single client. Data-stream clustering algorithms, which learn in a single pass over the data, align well with the one-shot federated learning paradigm, where a single communication round can be sufficient. The proposed Evolving Federated Density Peak Clustering (eFDPC) method combines evolving fuzzy data-stream clustering and density peak clustering in a federated setting. Each client independently evolves micro-clusters from private data mini-batches through cluster addition and merging. To limit communication, density peak clustering is applied to the learned micro-clusters and only the selected peaks are transmitted, rather than the full local models. A shuffler-based layer is introduced between the clients and the server to anonymize contributions prior to reconstruction and global peak selection. We consider a peak-only variant, eFDPC-P, as well as eFDPC-G with server-side label propagation. Experiments on synthetic and real-world benchmark datasets show that eFDPC outperforms recent federated clustering baselines in one-shot settings and scales to higher-dimensional, non-IID data.
Figures & tables
Figure 1: Overview of the proposed eFDPC pipeline. (1) Each client learns evolving micro-clusters from its private data using mini-batch updates. (2) Each client applies a local DPC step on its micro-clusters to select a small set of local peaks. (3) The selected peak statistics are split into additive shares and sent to multiple shufflers. Each shuffler permutes the received messages and forwards them to the server. (4) The server reconstructs the peak statistics by matching share identifiers and summing the shares, obtaining an unlinkable pool of candidate peaks. (5) The server performs a second DPC step on the reconstructed candidates to select the global peak set, and (6) broadcasts the global peaks back to clients for the next communication round.
Stage
Operation
Time
Memory
Client
Micro-cluster update (per batch)
O(NbCd)
O((C+Nb)d)
Micro-cluster update (full pass)
O(NCd)
Local DPC peak selection
O(C2d)
Share generation
O(SPd)
Communication
Per client, single round
O(Pd)
O(Pd)
Server
Peak reconstruction
O(SKPd)
O(KPd)
Table 1: Computational and memory complexity of eFDPC.
Figure 2: Comparison of eFDPC- G behavior on Skewed ( K=10 , left column) and MNIST ( K=3 , right column). From top to bottom, rows depict the decision graph (a,b), cluster grouping (c,d), and clustering accuracy (e,f) at the server-side. The evaluation is performed using the propagated labels on the shared peaks.
Table 4: Client-count K sweep for Dirichlet α=0.9 .
Figure 3: Mini-batch size sensitivity of eFDPC- G on MNIST for K=3,10,30 clients (Dirichlet α=0.9 ). ACC, ARI, and NMI are stable across the mini-batch size B , with a mild optimum at moderate sizes, and improve with the number of clients; runtime falls as B grows and grows with K . Bands show ±1 standard deviation over the repetitions.
Figure 4: Non-IID label partition of MNIST across K=10 clients with the Dirichlet split used in the experiments. Small α results in each true cluster being split between a few clients (a), while large α approaches an IID split where every client holds every cluster in similar proportions (d).
Clustering high-dimensional data is a fundamental task in unsupervised machine learning with applications to a variety of domains. In the centralized data scenario, this task is commonly solved using deep clustering methods that utilize deep neural network architectures to learn clustering-friendly latent space representations. In Federated Learning, where data is distributed between clients and is private, deep clustering methods are less explored. In particular, recently introduced federated deep clustering methods, despite showing very promising performance, still fall short in reliably providing good performance if data across clients are non-identically-independently distributed. In this work, we introduce a generalization of Deep Clustering Networks to the federated scenario, named FedDCN, that simultaneously optimizes a reconstruction loss and a clustering loss. To ensure robustness and latent space alignment in non-identically-independently distributed data scenarios, FedDCN generates synthetic data augmentations, and its learning objective includes a geometric regularization for latent space alignment. Through experimental evaluation, the effectiveness of the approach under IID and non-IID assumptions is demonstrated, and future research directions are identified.
Morris Stallmann, Charalampos S. Kouzinopoulos, Marcin Pietrasik +1
Department of Advanced Computing Sciences (DACS) Maastricht University Maastricht, Netherlands
Federated deep clustering seeks to learn clustering-friendly representations from decentralized unlabeled data while preserving client privacy. However, Deep Embedded Clustering (DEC)-style objectives depend on global soft-assignment statistics that require clients to reveal their sensitive information. We propose Fed-BRDECS, a privacy-preserving and heterogeneity-aware federated deep embedded clustering framework. Fed-BRDECS replaces the globally normalized clustering objective with a locally computable sample-stability loss, avoiding the transmission of local soft-assignment distributions. To tackle non-IID client distributions, we introduce prediction-balanced sampling, which oversamples locally rare predicted clusters without requiring ground-truth labels, and centroid-level restarting, which periodically refreshes biased or inactive centroids. Experiments on image and text clustering benchmarks show that Fed-BRDECS consistently outperforms representative federated clustering and deep clustering baselines under both IID and non-IID partitions. We further demonstrate its applicability to federated time-series anomaly detection, where it improves reconstruction-based detectors without adding inference-time cost.
Federated learning (FL) enables training of a global model while keeping raw data on end-devices. Despite this, FL has shown to leak private user information and thus in practice, it is often coupled with methods such as differential privacy (DP) and secure vector sum to provide formal privacy guarantees to its participants. In realistic cross-device deployments, the data are highly heterogeneous, so vanilla federated learning converges slowly and generalizes poorly. Clustered federated learning (CFL) mitigates this by segregating users into clusters, leading to lower intra-cluster data heterogeneity. Nevertheless, coupling CFL with DP remains challenging: the injected DP noise makes individual client updates excessively noisy, and the server is unable to initialize cluster centroids with the less noisy aggregated updates. To address this challenge, we propose PINA, a two-stage framework that first lets each client fine-tune a lightweight low-rank adaptation (LoRA) adapter and privately share a compressed sketch of the update. The server leverages these sketches to construct robust cluster centroids. In the second stage, PINA introduces a normality-driven aggregation mechanism that improves convergence and robustness. Our method retains the benefits of clustered FL while providing formal privacy guarantees against an untrusted server. Extensive evaluations show that our proposed method outperforms state-of-the-art DP-FL algorithms by an average of 2.9% in accuracy for privacy budgets (epsilon in {2, 8}).
Jie Xu, Haaris Mehmood, Rogier Van Dalen +2
Samsung R&D Institute UK (SRUK) · Samsung AI Centre Cambridge