Beyond Corner Patches: Semantics-Aware Backdoor Attack in Federated Learning

Kavindu Herath; Joshua Zhao; Saurabh Bagchi

cs.CRMar 31, 2026

Beyond Corner Patches: Semantics-Aware Backdoor Attack in Federated Learning

Authors: Kavindu Herath, Joshua Zhao, Saurabh Bagchi

Organizations: Department of Electrical and Computer Engineering, Purdue University, West Lafayette, USA

Abstract

Backdoor attacks on federated learning (FL) are most often evaluated with synthetic corner patches or out-of-distribution (OOD) patterns that are unlikely to arise in practice. In this paper, we revisit the backdoor threat to standard FL (a single global model) under a more realistic setting where triggers must be semantically meaningful, in-distribution, and visually plausible. We propose SABLE, a Semantics-Aware Backdoor for LEarning in federated settings, which constructs natural, content-consistent triggers (e.g., semantic attribute changes such as sunglasses) and optimizes an aggregation-aware malicious objective with feature separation and parameter regularization to keep attacker updates close to benign ones. We instantiate SABLE on CelebA hair-color classification and the German Traffic Sign Recognition Benchmark (GTSRB), poisoning only a small, interpretable subset of each malicious client's local data while otherwise following the standard FL protocol. Across heterogeneous client partitions and multiple aggregation rules (FedAvg, Trimmed Mean, MultiKrum, and FLAME), our semantics-driven triggers achieve high targeted attack success rates while preserving benign test accuracy. These results show that semantics-aligned backdoors remain a potent and practical threat in federated learning, and that robustness claims based solely on synthetic patch triggers can be overly optimistic.

Figures & tables

Fig. 1 : Overview of SABLE , our semantics-aware backdoor attack in federated learning. Top: Standard FL round with a mixture of benign and malicious clients. Benign clients train on clean local data, while malicious clients return poisoned updates that are aggregated by the server into the next global model. Bottom: Training pipeline on a malicious client. A subset of local images is used to generate semantic triggers, producing paired clean/triggered samples. The malicious client optimizes a joint objective that combines clean cross-entropy, triggered cross-entropy toward the target label, feature-separation loss on paired samples, and a regularization term that keeps its update close to the global model, yielding a poisoned local model that still appears benign under aggregation.

Panel of Figure 2 — Fig. 3 : Example CelebA samples used in SABLE . Top row: clean images with their original hair-color labels. Bottom row: corresponding triggered images where a natural semantic modification (sunglasses) is applied and the labels are remapped to the target class.

Panel of Figure 3 — Fig. 5 : Clean and triggered GTSRB stop–sign samples used in our experiments. Top row: clean STOP signs with their ground-truth label. Bottom row: triggered versions where a small blue cap symbol is placed above the sign, forming a semantics-aligned backdoor pattern that is relabeled to the attacker-chosen target class.

Aggregation/Defense	Benign-Only Accuracy	Baseline Attack [ 11 ]		SABLE (Us)
Aggregation/Defense	Benign-Only Accuracy	Acc	ASR	Acc	ASR
FedAvg	$86.00\pm 1.00$	$85.20\pm 0.74$	$77.20\pm 4.83$	$84.50\pm 1.20$	$\mathbf{81.30\pm 6.29}$
Trimmed Mean	$85.50\pm 2.01$	$87.50\pm 1.08$	$80.00\pm 3.46$	$85.10\pm 1.30$	$\mathbf{86.50\pm 5.97}$
MultiKrum	$87.50\pm 0.71$	$89.90\pm 1.30$	$52.10\pm 0.83$	$79.50\pm 2.11$	$\mathbf{84.50\pm 6.90}$
FLAME	$85.70\pm 2.19$	$81.20\pm 3.71$	$59.30\pm 10.36$	$83.70\pm 2.06$	$\mathbf{59.80\pm 7.70}$
FilterFL	$85.90\pm 2.38$	$86.30\pm 1.85$	$62.10\pm 4.76$	$82.60\pm 1.02$	$\mathbf{87.80\pm 2.71}$

TABLE I : Clean accuracy (Acc) and attack success rate (ASR) on the CelebA hair-color task for ResNet18 under different FL algorithms. Each cell reports two (Acc, ASR) pairs, formatted as mean

\pm

std, for the Bagdasaryan-style baseline and our proposed semantics-aware backdoor attack.

Aggregation/Defense	Benign-Only Accuracy	Baseline Attack [ 11 ]		SABLE (Us)
Aggregation/Defense	Benign-Only Accuracy	Acc	ASR	Acc	ASR
FedAvg	$84.30\pm 2.45$	$87.00\pm 0.82$	$91.60\pm 2.67$	$87.00\pm 0.82$	$\mathbf{91.70\pm 1.89}$
Trimmed Mean	$83.00\pm 0.81$	$80.50\pm 1.65$	$\mathbf{84.70\pm 2.45}$	$80.50\pm 1.08$	$83.60\pm 2.32$
MultiKrum	$81.60\pm 0.97$	$79.40\pm 2.55$	$79.10\pm 9.16$	$80.90\pm 1.79$	$\mathbf{94.50\pm 2.51}$
FLAME	$81.20\pm 3.71$	$81.30\pm 2.95$	$70.30\pm 15.00$	$79.50\pm 2.12$	$\mathbf{80.20\pm 4.18}$
FilterFL	$84.50\pm 1.35$	$91.30\pm 1.49$	$57.60\pm 7.85$	$90.80\pm 1.60$	$\mathbf{78.10\pm 10.40}$

TABLE II : Clean accuracy (Acc) and attack success rate (ASR) on the CelebA hair-color task for VGG-16 under different FL algorithms. Each cell reports two (Acc, ASR) pairs, formatted as mean

\pm

std, for the Bagdasaryan-style baseline and our proposed semantics-aware backdoor attack.

Aggregation/Defense	Benign-Only Accuracy	Baseline Attack [ 11 ]		SABLE (Us)
Aggregation/Defense	Benign-Only Accuracy	Acc	ASR	Acc	ASR
FedAvg	$96.80\pm 0.26$	$94.45\pm 7.97$	$96.60\pm 6.19$	$97.35\pm 0.34$	$\mathbf{100.00\pm 0.00}$
Trimmed Mean	$96.10\pm 0.21$	$97.05\pm 0.37$	$97.80\pm 1.48$	$96.60\pm 0.21$	$\mathbf{99.40\pm 0.97}$
MultiKrum	$96.30\pm 0.71$	$96.65\pm 0.47$	$63.40\pm 5.08$	$92.45\pm 1.48$	$\mathbf{98.40\pm 0.84}$
FLAME	$94.00\pm 1.11$	$96.55\pm 0.83$	$98.80\pm 1.93$	$96.65\pm 0.63$	$\mathbf{99.40\pm 0.97}$
FilterFL	$78.85\pm 0.84$	$73.80\pm 1.79$	$89.80\pm 2.01$	$72.85\pm 1.94$	$\mathbf{90.00\pm 1.79}$

TABLE III : Clean accuracy (Acc) and attack success rate (ASR) on the GTSRB traffic-sign task for VGG-16 under different FL algorithms. Each cell reports two (Acc, ASR) pairs, formatted as mean

\pm

std, for the Bagdasaryan-style baseline and our proposed semantics-aware backdoor attack.

Experiment	GPU	Malicious	Benign
CelebA - ResNet18	G4	$1.57\pm 0.33$	$0.58\pm 0.02$
	A100	$3.30\pm 0.34$	$1.59\pm 0.04$
	L4	$4.44\pm 0.36$	$2.03\pm 0.03$
CelebA - VGG16	G4	$3.16\pm 0.31$	$2.03\pm 0.01$
	A100	$4.74\pm 0.30$	$2.23\pm 0.04$
	L4	$12.32\pm 0.05$	$9.73\pm 0.04$

TABLE IV : Per-client training time for each epoch, mean

\pm

std (seconds), for malicious vs. benign clients across GPU types.

Panel of Figure 8 — Fig. 6 : Representation-level shift induced by SABLE versus the baseline. UMAP visualization of penultimate-layer embeddings from a ResNet-18 hair-color classifier on CelebA under MultiKrum training.

Fig. 7 : ASR of SABLE under varying malicious client ratios in a 10-client federated learning setup. We report mean ASR (%) for FedAvg, Trimmed Mean, and MultiKrum on CelebA with ResNet-18; error bars denote standard deviation across runs.

Panel of Figure 10 — Fig. 8 : Sensitivity analysis of the feature-separation weight ( $\lambda_{\mathrm{sep}}$ ) and regularization weight ( $\lambda_{\mathrm{reg}}$ ) for SABLE under Trimmed-Mean aggregation on CelebA with ResNet-18, measured in terms of clean accuracy (ACC) and attack success rate (ASR); error bars denote standard deviation across runs.

Explore similar work

Color Matters: Trigger Color Affects Success in Federated Backdoor Attacks
Jun 24, 2026Kavindu Herath, Joshua C. Zhao, Saurabh BagchiBackdoor Attacks Backdoor Attacks in Federated Learning
Department of Electrical and Computer Engineering, Purdue University, West Lafayette, USA
Mitigating Backdoor Attacks in Federated Learning Using PPA and MiniMax Game Theory
Mar 30, 2026Osama Wehbi, Sarhad Arisdakessian, Omar Abdel Wahab +3Backdoor Attacks in Federated Learning Federated Learning
Department of Computer and Software Engineering, Polytechnique Montréal, Montreal, Quebec, Canada · Institut national de la recherche scientifique (INRS-EMT), Montréal, Canada · Department of Computer Science, Khalifa University, Abu Dhabi, UAE +1
SCRUB-FL: Sanitizing and Cleansing Representations via Unlearning of Backdoors
Jun 21, 2026Osama Wehbi, Sarhad Arisdakessian, Omar Abdel Wahab +2Backdoor Attacks in Federated Learning Federated Learning
Department of Computer and Software Engineering, Polytechnique Montréal, Montreal, Quebec, Canada · Department of Computer Science, Khalifa University, Abu Dhabi, UAE

Jun 24, 2026J/K move · Enter open · S save

Color Matters: Trigger Color Affects Success in Federated Backdoor Attacks

Kavindu Herath, Joshua C. Zhao, Saurabh Bagchi

Department of Electrical and Computer Engineering, Purdue University, West Lafayette, USA

Backdoor Attacks Backdoor Attacks in Federated Learning Federated Learning

Open paper AskPDF arXiv

Federated learning is vulnerable to backdoor attacks in which malicious clients inject poisoned updates while preserving benign-task performance. In this paper, we study a semantics-driven backdoor mechanism in which attackers use natural visual accessories as triggers and manipulate only the trigger color while keeping the attack pipeline fixed. Our framework considers semantic trigger objects such as masks and sunglasses, instantiated in black and white variants, and evaluates their effect in a controlled federated learning setting. Malicious clients construct poisoned samples by applying a trigger to source-class images and relabeling them to an attacker-chosen target class, while benign clients train only on clean data. We analyze this mechanism under both a standard poisoning objective and a stronger SABLE-based objective that combines clean classification loss, triggered target loss, feature-separation loss in the penultimate representation space, and regularization to keep malicious updates close to the global model. This design enables the attack to remain effective while reducing excessive update drift. Experiments on a four-class CelebA hair-color task show that trigger color significantly changes attack success rate even when trigger semantics, placement, and poisoning budget are unchanged. White triggers are more effective for attacks targeting the blond class, whereas black triggers perform better for attacks targeting the black class. The same trend persists under robust aggregation, showing that trigger color is a meaningful factor in the operation, persistence, and evaluation of semantic backdoor mechanisms in federated learning.

Abstract

Explore similar work

Color Matters: Trigger Color Affects Success in Federated Backdoor Attacks

Mitigating Backdoor Attacks in Federated Learning Using PPA and MiniMax Game Theory

SCRUB-FL: Sanitizing and Cleansing Representations via Unlearning of Backdoors