Big data scenarios, where massive, heterogeneous datasets are distributed across clients, demand scalable, privacy-preserving learning methods. Federated learning (FL) enables decentralized training of machine learning (ML) models across clients without data centralization. Decentralized training, however, introduces a computational burden on client devices. U-shaped federated split learning (UFSL) offloads a fraction of the client computation to the server while keeping both data and labels on the clients' side. However, the intermediate representations (i.e., smashed data) shared by clients with the server are prone to exposing clients' private data. To reduce exposure of client data through intermediate data representations, this work proposes k-anonymous differentially private UFSL (KD-UFSL), which leverages privacy-enhancing techniques such as microaggregation and differential privacy to minimize data leakage from the smashed data transferred to the server. We first demonstrate that an adversary can access private client data from intermediate representations via a data-reconstruction attack, and then present a privacy-enhancing solution, KD-UFSL, to mitigate this risk. Our experiments indicate that, alongside increasing the mean squared error between the actual and reconstructed images by up to 50% in some cases, KD-UFSL also decreases the structural similarity between them by up to 40% on four benchmarking datasets. More importantly, KD-UFSL improves privacy while preserving the utility of the global model. This highlights its suitability for large-scale big data applications where privacy and utility must be balanced.
Figures & tables
Figure 1: The clients’ data on the top, and reconstructed data from the client’s smashed data at the bottom.
Not.
Explanation
Not.
Explanation
C
Set of participating clients
G
Set of client groups
n
Number of participating clients
k
Number of groups for k-anonymity
Dc
Private dataset at client c∈C
D
Complete dataset in all clients
Fc(w)
Local objective function for client c∈C
wct
Local model weights for client c∈C in iteration t
Hc
Head network for client c∈C
wH
Weights for the head network
Bc
Body network for client c∈C
wB
Weights for the body network
Table I : Notations used in this paper
Figure 2: The process of training UFSL with model-level k-anonymity or micro-aggregation.
Dataset
No. of samples
No. of classes
Fraction used
CIFAR10
60,000
10
1.0
EMNIST
800,000
26
0.1
FashionMNIST
70,000
10
0.4
SVHN
600,000
10
0.1
Table II : Datasets used in this work.
Figure 3: ResNet-18 architecture split into head, body, and tail networks alongside the inversion network. RB1 refers to ResBlock1 and so on.
Network
Method
Metric
CIFAR10
EMNIST
FashionMNIST
SVHN
ResNet18
UFSL
MSE ↑
0.007
0.344
0.655
0.046
SSIM ↓
0.949
0.159
0.212
0.846
UFSL+DP σ2=0.2
MSE ↑
0.010
0.302
0.403
0.054
SSIM ↓
0.922
0.240
0.176
0.758
UFSL+KA k=3
MSE ↑
0.035
0.417
0.881
0.047
SSIM ↓
0.752
0.164
0.150
0.852
Table III : Averaged test dataset per image MSE and SSIM between actual and reconstructed data for KD-UFSL and baselines, across different networks. (Bold results represent the best value over the network and the dataset.)
Figure 4: Actual images and reconstructed images from the client’s smashed data of different training strategies, using the inversion network. Each row represents a single model.
Figure 5: Accuracy over global rounds of the final global model for KD-UFSL and baselines. CN refers to ConvNet, and RN18 refers to ResNet18.
Figure 6: Impact of σ2 on MSE and SSIM every 5th global iteration on the ConvNet architecture.
Figure 7: Impact of k on MSE and SSIM every 5th global iteration on the ConvNet architecture.
Head Depth
CIFAR10
SVHN
MSE ↑
SSIM ↓
MSE ↑
SSIM ↓
RB1
0.045
0.762
0.057
0.882
RB2
0.086
0.547
0.099
0.572
RB3
0.313
0.035
0.274
0.043
Table IV : Impact of head network depth on KD-UFSL performance. Depth up to layer RBi also includes layer RBi.
Figure 8: Impact of number of clients on SSIM, MSE, and accuracy in the KD-UFSL framework with the ConvNet architecture.
One-shot federated learning (OSFL) addresses the communication overhead of federated learning by limiting training to a single round, but doing so without sacrificing model quality is non-trivial, particularly when client data distributions diverge. Recent work has addressed this challenge by aggregating client knowledge on the server through the construction of transferable synthetic datasets or distillates. However, most of these methods lack formal privacy guarantees, leaving a gap in jointly achieving low communication, robustness to heterogeneity, and rigorous privacy. We propose FedKT-CSD (Federated Knowledge Transfer via Collaborative Synthetic Data), a framework inspired by neural image compression that closes this gap by leveraging publicly pretrained autoencoders as a shared latent space. Each client encodes its private data in a single forward pass, computes class-conditional latent statistics, and transmits these to the server. The server aggregates these statistics via secure aggregation, adds calibrated differential privacy noise, and decodes a synthetic dataset for training a global model and further downstream tasks. This design provides formal (ε,δ)-differential privacy by construction, while keeping client-side computation and communication lightweight. Despite operating under privacy constraints, FedKT-CSD is competitive with and even outperforms non-private baselines across diverse datasets and heterogeneity settings, and scales to a large number of clients. Our code is available at: https://github.com/an7123/FedKT-CSD
Maximilian Andreas Hoefler, Karsten Mueller, Wojciech Samek
Fraunhofer Heinrich Hertz Institute · BIFOLD Technical University Berlin
Split Federated Learning (SFL) enables privacy-preserving collaborative training by partitioning models between clients and a server. However, under non-IID data distributions, SFL often suffers from biased optimization and unstable convergence, while existing solutions largely adapt techniques from conventional federated learning. In this work, we observe that the split architecture of SFL inherently alters how client information is represented and coordinated, opening opportunities for bias compensation beyond parameter-level aggregation. Based on this insight, we propose BESplit, an architecture-aware framework that exploits the intrinsic structure of SFL to mitigate non-IID effects. First, to prevent biased local data from dominating global updates, we introduce Evidential Aggregation (EA) to perform fine-grained reweighting of client contributions based on evidential uncertainty. Second, to further reduce distributional skew, we develop Bias-Compensated Collaboration (BCC) to align split-layer representations by pairing complementary clients. Finally, Dual-Teacher Distillation (DTD) is incorporated to synchronize knowledge between decoupled client and server models, enabling independent local inference. Extensive experiments on five benchmark datasets demonstrate that BESplit consistently outperforms state-of-the-art methods in accuracy, convergence stability, and computational efficiency under diverse non-IID settings.
Yuhan Xie, Chen Lyu, Jingrong Huang
Shanghai University of Finance and Economics, China · MoE Key Laboratory of Interdisciplinary Research of Computation and Economics, China.
We propose a novel end-to-end privacy-preserving framework, instantiated by three efficient protocols for different deployment scenarios, covering both input and output privacy, for the vertically split scenario in federated learning (FL), where features are split across clients and labels are not shared by all parties. We do so by distributing the role of the aggregator in FL into multiple servers and having them run secure multiparty computation (MPC) protocols to perform model and feature aggregation and apply differential privacy (DP) to the final released model. While a naive solution would have the clients delegating the entirety of training to run in MPC between the servers, our optimized solution, which supports purely global and also global-local models updates with privacy-preserving, drastically reduces the amount of computation and communication performed using multiparty computation. The experimental results also show the effectiveness of our protocols.