Federated Art Chase: Privacy Leakage and Data Poisoning in Federated Fine-Tuning

Vasileios Perifanis; Nikolaos Pavlidis

cs.LGOct 15, 2026

Federated Art Chase: Privacy Leakage and Data Poisoning in Federated Fine-Tuning

Authors: Vasileios Perifanis, Nikolaos Pavlidis

Organizations: Indigma Innovations Greece

Abstract

Federated learning (FL) is increasingly used to adapt a pretrained, centrally fine-tuned model to data that remain on user devices, beyond training a model from scratch. In this report, prepared for the 4th Federated Machine Learning International Summer School, we study the privacy and integrity of this approach with a reproducible pipeline. An ImageNet-pretrained MobileNetV4-Conv-Small model is fine-tuned centrally on ten Quick, Draw! classes (97.50% test accuracy) and then adapted with head-only FedAvg over ten non-IID clients, which raises client training accuracy from 90.34% to 91.49% while preserving global accuracy. We evaluate four adversarial scenarios. First, from a single one-step client update, an honest-but-curious server recovers the complete private label set, but its reconstructions follow class prototypes rather than the private drawings (mean SSIM 0.130). Second, from a one-round FedAvg aggregate, label-count recovery falls from 87.5% with one client to 75.0% with five and ten clients, below the 80.0% of an uninformed uniform guess. Third, under FedAvg, a backdoor injected by 30%, 50%, and 100% of the clients reaches an attack success rate (ASR) of 21.9%, 31.3%, and 54.7% at a cost of at most 0.8 percentage points of clean accuracy, whereas label flipping is effective only when most of the source-class data are relabeled. Fourth, FLTrust with a root set retained from centralized fine-tuning lowers the backdoor ASR to 1.8% and 4.0% at 30% and 50% malicious clients and the label-flipping ASR from 12.6% to 0.8% without measurable clean-accuracy loss, but provides no protection when every client is malicious. The report closes with a live session in which participants draw, fine-tune, and poison the model through Indigma's FedMaestro orchestrator. Code and executed notebooks, runnable on Google Colab, are at https://github.com/Indigma-Innovations/fl-summer-school.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the pipeline. Blue boxes denote model development stages, orange boxes privacy attacks by an honest-but-curious server, red and green boxes poisoning by malicious clients and the FLTrust defense, and the gray box the live session. NB $k$ denotes the executed notebook that implements each component. Attacks 1 and 2 start from the Stage 1 checkpoint, whereas Attack 3 and the defense reuse the exact client population of Stage 2.

Experiment	Adversary	Capabilities and assumptions	Reported metrics
Stage 1 (NB1)	none	centralized training on QuickDraw-10	loss, accuracy
Stage 2 (NB4)	none	10 clients, head-only FedAvg	loss, accuracy (client and global)
Attack 1 (NB2)	honest-but-curious server	observes $\theta_{t}$ , one client delta $\Delta_{k}$ , $n_{k}$ , $\eta$ ; public QuickDraw data	label-set accuracy; MSE, PSNR, SSIM
Attack 2 (NB3)	honest-but-curious server	observes $\theta_{t}$ , $\theta_{t+1}$ , $m$ , $N=\sum_{k}n_{k}$ , $\eta$ ; public QuickDraw data	label-count accuracy; MSE, PSNR, SSIM
Attack 3 (NB5)	malicious clients (fraction $f$ )	poison local training data; follow the protocol otherwise	clean accuracy, ASR
Defense (NB6)	malicious clients (fraction $f$ )	as Attack 3; server holds a clean root set	clean accuracy, ASR, trust scores

Table 1: Experiments, adversaries, and reported metrics.

\theta_{t}

denotes the global classifier head at round

t

\Delta

a model update,

n_{k}

the sample count of client

k

, and

\eta

the local learning rate.

Figure 2: Five randomly drawn Stage 1 examples per class ( $28\times 28$ bitmaps, white strokes on a black background).

Use	Indices	Size	Selection
Stage 1 train / validation / test (NB1)	$0$ – $4{,}999$	40,000 / 5,000 / 5,000	stratified 80/10/10 split per class (seed 42)
Federated clients (NB4–NB6)	$\geq 10{,}000$	13,675 (12,310 train, 1,365 val.)	hard selection from $\geq 5{,}000$ random candidates per class
Attack 1 private batch (NB2)	$\geq 20{,}000$	5	hardest of 128 random candidates per class
Attack 2 private pool (NB3)	$\geq 20{,}000$	56 (10 clients)	hardest of 128 random candidates per class and client
Attacker public priors (NB2, NB3)	$<20{,}000$	96 per class (prototypes), 1 per canvas	uniform random
Global evaluation set (NB4–NB6)	$0$ – $4{,}999$	5,000	test part of the 80/10/10 split rebuilt with seed 123 (79.8% overlap with the Stage 1 training split)

Table 2: Data provenance. Indices refer to positions within each QuickDraw class file. “Hard” selection retains the candidates with the highest cross-entropy loss under the Stage 1 model (Sections 5 and 6 ).

Split	Loss before	Loss after	Accuracy before (%)	Accuracy after (%)	$\Delta$ accuracy (pp)
Train (augmented)	7.8110	0.2378	6.97	92.29	$+85.32$
Validation	8.5807	0.0961	5.64	97.48	$+91.84$
Test	8.5695	0.0969	6.00	97.50	$+91.50$

Table 3: Stage 1 performance before fine-tuning (pretrained backbone, untrained head) and after fine-tuning (selected checkpoint, epoch 15).

Figure 3: Stage 1 learning curves. (a) Cross-entropy loss and (b) accuracy per epoch.

Figure 4: Client label distribution of the federated population. Each cell gives the number of local samples (training and validation) of a class, and gray cells mark classes that a client does not hold. Classes per client range from 7 to 10 and dataset sizes from 1,015 to 1,890 samples.

View	Loss	Accuracy (%)	Loss	Accuracy (%)	$\Delta$ accuracy (pp)
	Round 0		Round 50
Client train (12,310)	0.3876	90.34	0.3703	91.49	$+1.15$
Client validation (1,365)	0.4305	89.67	0.4364	89.89	$+0.22$
Global evaluation (5,000)	0.0925	97.66	0.0910	97.64	$-0.02$

Table 4: Clean head-only federated fine-tuning before the first round and after 50 FedAvg rounds. Client metrics are sample-weighted over all ten clients.

Figure 5: Clean federated fine-tuning over 50 rounds. Columns show the pooled client training and validation data, and the global evaluation set, while rows show cross-entropy loss and accuracy.

Figure 6: Attack 1: private samples (top) and reconstructions from the single-client update using the recovered labels (middle) and the true labels (bottom, evaluation-only diagnostic).

Figure 7: Label-leakage signal in a single head update. (a) Bias gradient

g_{b}^{(c)}

per class. Asterisks mark the five most negative entries selected by the naive rule, which includes the absent class sun and misses line . (b) Norm of the weight-gradient row

g_{W}^{(c)}

on a logarithmic scale. Both circle (present) and donut (absent) have large norms because the private circle sample is confidently predicted as donut .

Rule	Recovered label set	Correct	Margin
Top- $n$ most negative $g_{b}^{(c)}$ ( $\equiv$ Eq. ( 5 ) with $\lambda=0$ )	square , triangle , circle , star , sun	4/5	0.0011
Set search, Eq. ( 5 ) with $\lambda=0.01$	line , circle , square , triangle , star	5/5	0.0133

Table 5: Attack 1: label-set recovery from one single-client update. The margin is the score difference to the second-ranked set.

	Recovered labels			Oracle labels
Class	MSE $\downarrow$	PSNR (dB) $\uparrow$	SSIM $\uparrow$	MSE $\downarrow$	PSNR (dB) $\uparrow$	SSIM $\uparrow$
line	7690.18	9.27	0.069	7620.23	9.31	0.070
circle	11111.11	7.67	0.208	10979.75	7.72	0.190
square	8178.44	9.00	0.192	8155.11	9.02	0.191
triangle	7658.22	9.29	0.010	7478.31	9.39	0.010
star	10332.21	7.99	0.171	10327.46	7.99	0.172

Table 6: Attack 1: reconstruction quality against the private

28\times 28

sketches (

\downarrow

: lower is better,

\uparrow

: higher is better). Rows marked

\dagger

are post hoc references (Section 6.2 ).

$m$	Clients	$N$	Correct	Count accuracy (%)	Uniform (%)	$J(\tilde{q})$	Classes at bound $m$
1	0	8	7/8	87.5	80.0	10.84	– ∗
2	0–1	12	10/12	83.3	73.3	10.45	5
5	0–4	28	21/28	75.0	80.0	10.20	5
10	0–9	56	42/56	75.0	80.0	10.44	5
∗ For $m=1$ , every non-zero count equals the bound.

Table 7: Attack 2: label-count recovery from a one-round FedAvg aggregate (Notebook 3). “Uniform” is the accuracy of the uninformed allocation

\hat{q}_{c}=N/C

, and

J(\tilde{q})

is the objective of Eq. ( 7 ) at the continuous solution, to be compared with

J=11

for

q=0

Figure 8: Attack 2: true and recovered label counts for

m=1,2,5,10

participating clients (Notebook 3). Shaded columns mark the classes withheld from all clients. The dashed line is the per-class upper bound

q_{c}\leq m

. For

m\geq 2

, the solution places five classes at the bound and omits classes that occur several times.

$m$	Pairs	MSE $\downarrow$	PSNR (dB) $\uparrow$	SSIM $\uparrow$	MSE $\downarrow$	PSNR (dB) $\uparrow$	SSIM $\uparrow$	Runtime (s)
		Matched pairs			Full (missed $\to$ empty canvas)
1	7/8	8615.64	8.81	0.212	8034.18	9.23	0.186	393.9
2	10/12	10081.14	8.17	0.117	10285.06	8.08	0.097	86.0
5	21/28	8087.35	9.32	0.198	8281.28	9.18	0.148	53.1
10	42/56	8672.74	8.99	0.219	8611.48	9.07	0.164	33.9

Table 8: Attack 2: reconstruction quality against the private sketches. “Matched” averages over reconstructions assigned to a private sample of the same class, while “Full” also includes each missed private sample, scored against an empty canvas.

Figure 9: Attack 2: private samples (upper row of each block) and reconstructions from the FedAvg aggregate (lower row), matched within each class by minimum MSE. The first block uses the true label counts for $m=1$ (evaluation only), and the others use the recovered counts.

Figure 10: Poisoning mechanisms illustrated on Stage 1 sketches. (a) Label flipping changes only the label of circle samples to donut . (b) Backdoor poisoning adds a fixed trigger to 20% of the training samples of a malicious client, drawn from its non- donut samples, and relabels them as donut . (c) A 32-pixel trigger pattern.

Attack	$f$	Malicious clients	Weight	Poisoned	Acc.	ASR (%)	Acc.	ASR (%)
					FedAvg		FLTrust
Label flipping	0.0	–	0.0%	0 (0.0%)	97.62	0.60	97.66	0.60
	0.1	2	9.8%	31 (2.2%)	97.64	0.60	97.64	0.60
	0.3	4, 5, 8	34.3%	888 (64.3%)	96.48	12.60	97.64	0.80
	0.5	0, 1, 2, 8, 9	50.1%	296 (21.4%)	97.68	0.60	97.66	0.60
	1.0	all	100.0%	1,381 (100%)	87.92	97.80	88.02	97.80

Table 9: Poisoning composition and final results after 50 rounds under FedAvg and FLTrust. “Weight” is the share of all client training samples held by malicious clients, i.e., their total FedAvg weight. “Poisoned” counts relabeled circle samples for label flipping (share of all 1,381 circle training samples) and triggered samples for the backdoor (share of all 12,310 training samples). Acc. denotes clean accuracy on the global evaluation set (%).

Figure 11: Label flipping ( circle

\to

donut ) over 50 rounds under FedAvg (left) and FLTrust (right). The top row shows ASR on clean circle evaluation images, and the bottom row clean accuracy on the global evaluation set. Color encodes the malicious fraction

f

, and the dashed gray line is the

f=0

control. End labels give the final ASR where it exceeds 5%. Under FedAvg, only

f=0.3

(64% of circle data relabeled) and

f=1.0

(100%) have an effect. FLTrust removes the

f=0.3

effect and delays the

f=1.0

attack (80% ASR at round 9 instead of round 7).

Figure 12: Final ASR after 50 rounds as a function of the poisoned data mass. (a) Label flipping against the share of circle training samples that are relabeled. The non-nested malicious sets place

f=0.5

to the left of

f=0.3

. (b) Backdoor against the share of all client training samples that are triggered. Labels give the malicious fraction

f

of each FedAvg point.

Figure 13: Backdoor poisoning over 50 rounds under FedAvg (left) and FLTrust (right). The top row shows ASR on triggered non- donut evaluation images, and the bottom row clean accuracy on the global evaluation set. Under FedAvg, ASR grows steadily with

f

, while final clean accuracy stays within 0.8 pp of the control (with a transient dip of 1.8 pp at

f=1.0

). FLTrust keeps ASR below 4.1% for

f\leq 0.5

, but not for

f=1.0

Figure 14: FLTrust trust scores (Eq. ( 8 )) per round, averaged over honest (solid) and malicious (dashed) clients, for label flipping (top) and the backdoor (bottom). At

f=1.0

there are no honest clients, and the dotted line shows the honest trust of the

f=0

run for reference.

Attack	$f$	Honest	Malicious	FedAvg weight	FLTrust share	ASR FedAvg (%)	ASR FLTrust (%)
		Mean trust score		Malicious aggregation weight
Label flipping	0.0	0.384	–	0.0%	0.0%	0.60	0.60
	0.1	0.387	0.234	9.8%	6.3%	0.60	0.60
	0.3	0.413	0.227	34.3%	19.0%	12.60	0.80
	0.5	0.429	0.285	50.1%	39.8%	0.60	0.60
	1.0	–	0.230	100.0%	100.0%	97.80	97.80

Table 10: FLTrust trust diagnostics averaged over 50 rounds. “FedAvg weight” is the aggregated weight of malicious clients under Eq. ( 2 ). “FLTrust share” is their share of the total trust,

\sum_{k\in\mathcal{M}}s_{k}/\sum_{k}s_{k}

, computed per round from the mean honest and malicious trust and averaged over rounds.

Notebook	Operation	Runtime	Peak RAM (MiB)	Peak VRAM (MiB)
NB1	Centralized fine-tuning, 15 epochs of 40,000 images	56.9 min	n/r	n/r
NB2	One head-only SGD step on 5 images (client side)	0.086 s	1288.1	35.0
NB3	Attack 2, complete experiment, $m=1$ / 2 / 5 / 10	393.9 / 86.0 / 53.1 / 33.9 s	1689–1782	100–571
NB4	Clean FedAvg, 50 rounds, 10 clients (training and aggregation)	12.55 s	2333.5	31.3
NB5	FedAvg under poisoning, per 50-round scenario	11.19–12.10 s	$\leq$ 2444.7	31.3
NB6	FLTrust under poisoning, per 50-round scenario	12.03–12.70 s	$\leq$ 2448.9	31.3

Table 11: Runtime and peak memory as recorded by the notebooks on a Google Colab GPU runtime. RAM is the peak resident set size of the Python process, and VRAM is the peak memory allocated by PyTorch. Federated runtimes exclude data loading, hard-example selection, and the one-time embedding precomputation. n/r: not recorded.

Figure 15: Participant view of the live session. The participant selects a class, draws it on the canvas, and receives the five most probable classes under the current model served by the orchestrator, together with their confidences.

Figure 16: Round progress in the FedMaestro administrator dashboard. Clean rounds (Step 2) and poisoning rounds (Step 4) are listed separately with their status, the number of received updates, and the global loss reported by the orchestrator. The screenshot was taken during a test run with a single client.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 951 papers

May 4, 2026

Adversarial Update-Based Federated Unlearning for Poisoned Model Recovery

Federated learning (FL) is vulnerable to poisoning attacks, where malicious clients upload manipulated updates to degrade the performance of the global model. Although detection methods can identify and remove malicious clients, the model remains affected. Retraining from scratch is effective but costly, and existing unlearning methods remain unsatisfactory in both effectiveness and efficiency. We propose Federated Adversarial Unlearning (FAUN), a lightweight framework that retains only a short window of malicious clients' updates and employs adversarial optimization on a proxy dataset to derive updates that eliminate malicious directions. Applying these updates for a few unlearning rounds, followed by benign fine-tuning, enables fast removal of malicious effects and stable recovery. Experiments on three canonical datasets show that FAUN achieves recovery comparable to retraining while requiring far fewer rounds and reduces attack success rates to near zero, confirming FAUN successfully eliminates the contributions of unlearned clients.

Wenwei Zhao, Xiaowen Li, Yao Liu +1

Department of Electrical Engineering, University of South Florida,Tampa, FL, USA · Bellini College, University of South Florida, Tampa, FL, USA

Federated Unlearning Machine Unlearning Federated Learning

Ask PDF

Mar 30, 2026

FedFG: Privacy-Preserving and Robust Federated Learning via Flow-Matching Generation

Federated learning (FL) enables distributed clients to collaboratively train a global model using local private data. Nevertheless, recent studies show that conventional FL algorithms still exhibit deficiencies in privacy protection, and the server lacks a reliable and stable aggregation rule for updating the global model. This situation creates opportunities for adversaries: on the one hand, they may eavesdrop on uploaded gradients or model parameters, potentially leaking benign clients' private data; on the other hand, they may compromise clients to launch poisoning attacks that corrupt the global model. To balance accuracy and security, we propose FedFG, a robust FL framework based on flow-matching generation that simultaneously preserves client privacy and resists sophisticated poisoning attacks. On the client side, each local network is decoupled into a private feature extractor and a public classifier. Each client is further equipped with a flow-matching generator that replaces the extractor when interacting with the server, thereby protecting private features while learning an approximation of the underlying data distribution. Complementing the client-side design, the server employs a client-update verification scheme and a novel robust aggregation mechanism driven by synthetic samples produced by the flow-matching generator. Experiments on MNIST, FMNIST, and CIFAR-10 demonstrate that, compared with prior work, our approach adapts to multiple attack strategies and achieves higher accuracy while maintaining strong privacy protection.

Ruiyang Wang, Rong Pan, Zhengan Yao

School of Mathematics, Sun Yat-sen University, Guangzhou 510275, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, China · Institute of Advanced Studies Hong Kong, Sun Yat-sen University, Hong Kong

Privacy-Preserving Federated Learning Federated Learning Byzantine-Robust Federated Learning

Ask PDF

Jun 10, 2026

Fed-FBD: Federated Functional Block Diversification for Isolation, Privacy, and Surgical Unlearning

Federated learning (FL) enables collaborative model training without sharing raw patient data, but standard approaches such as FedAvg treat each client as a black box and provide no mechanism for isolating an adversarial contributor, auditing per-client influence, or honoring a departed participant's right to be forgotten. We present Fed-FBD (Federated Functional Block Diversification), a modular federated architecture that decomposes a ResNet backbone into six functional blocks (the stem, four residual groups, and the classification head) and maintains a warehouse of N color variants, each assembled from independently tracked and contributor-stamped blocks. Fed-FBD provides three capabilities absent in FedAvg: (i) architecturally guaranteed block-level isolation, so that an adversarial or mislabelled client cannot contaminate the clean colous; (ii) privacy-by-design, where membership inference advantage is already indistinguishable from chance before any privacy mechanism is applied; and (iii) surgical machine unlearning of a departed participant's contribution at sub-second cost and without retraining. Experiments on six MedMNIST-2D datasets, PathMNIST at 224x224, and CIFAR-10 show that Fed-FBD trades a modest 0.3%-3.1% IID accuracy gap on the adequately sized datasets for these guarantees, remains within 0.8%-4.0% of FedAvg at Dirichlet alpha=1.0 on three of four datasets, and confines all six adversarial attacks we study to the poisoned client's own blocks with at most +/-0.01 AUC drift on the clean colors.

Weijie Chen, Alan B. McMillan

Department of Radiology, University of Wisconsin–Madison

Federated Unlearning Byzantine-Robust Federated Learning Privacy-Preserving ML Federated Learning Privacy-Preserving Federated Learning

Ask PDF

Abstract

Federated Learning

Explore similar work

Adversarial Update-Based Federated Unlearning for Poisoned Model Recovery

FedFG: Privacy-Preserving and Robust Federated Learning via Flow-Matching Generation

Fed-FBD: Federated Functional Block Diversification for Isolation, Privacy, and Surgical Unlearning