Why Backdooring Neural Networks is so Easy?

Issam Seddik; Mohamed El Amine Seddik

cs.LGSep 28, 2026

Why Backdooring Neural Networks is so Easy?

Authors: Issam Seddik, Mohamed El Amine Seddik

Organizations: Université Paris-Saclay, CEA LIST Palaiseau, France · AI Research Center, Technology Innovation Institute (TII) Abu Dhabi, UAE

Abstract

Securing modern AI systems against backdoor attacks remains an open challenge and requires fundamentally principled estimates of the adversary's budget -- the poison fraction $π$ and trigger strength $α$ needed to construct successful yet stealthy attacks. Motivated by recent empirical evidence that poisoning large language models can require a nearly constant number of malicious samples even as clean datasets grow, we derive an exact closed-form analysis of a quadratic neuron trained on a poisoned Gaussian mixture. We show, perhaps counterintuitively, that the same feature-learning dynamics that make neural networks powerful can also make them more vulnerable to backdoors. Specifically, with clean accuracy preserved to first order, $O(π)$ , we demonstrate that lazy learning imposes the inverse-square-root scaling $α\propto π^{-1/2}$ for a successful attack, while feature learning induces a quadratic detector whose loss margin scales as $O(α^4)$ , improving the attack budget to $α\propto π^{-1/4}$ . Consequently, nonlinear feature learning substantially reduces the trigger strength required at small poison fractions, thereby in a sense making feature learners more backdoor vulnerable. These results provide a theoretical mechanism consistent with large-scale empirical observations and demonstrate that security audits based on linear heuristics can systematically underestimate backdoor vulnerability in the widely adopted feature-learning regimes.

Figures & tables

Figure 1: Feature learning lowers the adversary’s budget. Trigger strength

\alpha

required to reach an attack success rate of

0.9

(top) and

0.6

(bottom) as a function of the poison fraction

\pi

, for a network trained in the lazy regime (frozen features) and in the rich regime (feature learning), on synthetic Gaussian data, MNIST, Fashion-MNIST and CIFAR-10; shaded areas are standard deviations over runs. Feature learning requires a smaller trigger at every poison fraction.

Figure 2: Exact theory in the two regimes.

\mathrm{CACC}

(triangles) and

\mathrm{ASR}

(circles) from Theorem 1 as functions of

\pi

, for

\alpha=2

and

\alpha=5

, in the feature-learning regime (

\bm{u}=\bm{u}^{\star}

, solid) and the linear regime (

\beta\equiv 0

, dashed);

S=1.5

T=1

\rho=0

. In both regimes the clean accuracy stays at

\Phi(S)\approx 0.93

, while the attack success rate is uniformly higher with feature learning and, at

\alpha=5

, exceeds

0.5

at a poison fraction roughly ten times smaller than in the linear regime, consistent with ( 15 ) .

	Linear regime (Thm. 2 )	Feature learning (Thm. 3 )
Trained curvature	$\beta\equiv 0$	$\beta=-\pi\frac{1+2S^{2}}{2(1+S^{2})}A^{2}+O(\pi^{2})$
Trigger term in the margin	$\propto\pi A^{2}$	$\tfrac{\beta}{2}A^{2}\propto\pi A^{4}$
Conserved invariant	$e=\pi\alpha^{2}T^{2}$	$\varrho=\pi\alpha^{4}T^{4}$
Stealth-limit ASR	$\Phi\!\bigl(-S+\tfrac{1+2S^{2}}{S}\tfrac{e}{e+2}\bigr)$	$\Phi\!\bigl(-S+\tfrac{1+2S^{2}}{S}\tfrac{\varrho}{4+\varrho}\bigr)$
Ceiling ( $\pi\to 0$ )	$\Phi\!\bigl(\tfrac{1+S^{2}}{S}\bigr)$	$\Phi\!\bigl(\tfrac{1+S^{2}}{S}\bigr)$
Budget curve $\alpha(\pi;a)$	$\tfrac{1}{T}\bigl(\tfrac{2\kappa_{a}}{(1-\kappa_{a})\pi}\bigr)^{1/2}$	$\tfrac{1}{T}\bigl(\tfrac{4\kappa_{a}}{(1-\kappa_{a})\pi}\bigr)^{1/4}$

Table 1: Linear regime versus feature-learning regime (hidden trigger,

\rho=0

;

A=\alpha T

;

\kappa_{a}=S(S+\Phi^{-1}(a))/(1+2S^{2})

). The two regimes share the clean accuracy, the form of the ASR law and the stealth ceiling; they differ in the conserved trigger energy and hence in the cost exponent.

Figure 3: Theory versus practice on synthetic and real data. Trigger strength

\alpha

required to reach

\mathrm{ASR}=0.6

(top) and

\mathrm{ASR}=0.9

(bottom) as a function of

\pi

. Markers are trained models (feature-learning regime in red, linear regime in blue) and lines the predictions of Theorem 1 on the invariants extracted from the data. The quadratic neuron follows the

\alpha\propto\pi^{-1/4}

law, the linear model the steeper

\alpha\propto\pi^{-1/2}

law.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Two backdoor channels. (a) The $(\pi,\,h/T)$ plane splits at the crossover $h=T\pi^{1/4}$ (dashed blue) into the feature-learning channel, $\alpha\propto\pi^{-1/4}$ , and the linear channel, $\alpha\propto\pi^{-1/2}$ (at $h=0$ , $\beta\equiv 0$ ). Grey contours are lines of constant exact required trigger strength from Theorem 1 . (b) Log-log plot of $\alpha(\pi;a)$ for the two extreme alignments, confirming the two exponents. Parameters: $S=1.5$ , $\rho=0$ , target ASR $a=0.6$ .

		Rich Regime						Lazy Regime
Dataset	$\pi$	$\alpha=1.0$		$\alpha=3.0$		$\alpha=5.0$		$\alpha=1.0$		$\alpha=3.0$		$\alpha=5.0$
		CACC	ASR	CACC	ASR	CACC	ASR	CACC	ASR	CACC	ASR	CACC	ASR
	0.01	0.93	0.08	0.93	0.30	0.93	0.70	0.93	0.08	0.93	0.14	0.93	0.33
Synthetic	0.02	0.93	0.08	0.93	0.47	0.93	0.84	0.93	0.09	0.93	0.21	0.93	0.52
	0.05	0.93	0.10	0.93	0.67	0.93	0.92	0.93	0.12	0.92	0.43	0.93	0.74
	0.01	0.97	0.97	0.97	0.99	0.97	1.00	0.85	0.15	0.85	0.28	0.85	0.59

Appendix

Table 2: Clean Accuracy (CACC) and Attack Success Rate (ASR) for different values of poison fraction (

\pi

) and trigger strength (

\alpha

Backdoor Attacks

Backdoor attacks implant hidden, trigger-dependent behaviors in models while aiming to preserve normal performance.

See all 135 papers

Poisoning-Based Backdoors Are Not Cryptographic
Oct 20, 2026Masahiro KanekoBackdoor Attacks LLM Backdoor Attacks
MBZUAI
When Stronger Triggers Backfire: A High-Dimensional Theory of Backdoor Attacks
May 21, 2026Donald Flynn, Hadas Yaron Goldhirsh, Jonathan P. Keating +1Backdoor Attacks Statistical Learning Theory
1Mathematical Institute, University of Oxford · School of Mathematical Science, Tel Aviv University · School of Mathematical Science and Computer Science, Tel Aviv University
Statistically Undetectable Backdoors in Deep Neural Networks
Jul 10, 2026Andrej Bogdanov, Alon Rosen, Neekon VafaBackdoor Attacks Adversarial Examples
University of Ottawa. · Bocconi University. · Massachusetts Institute of Technology.

Oct 20, 2026J/K move · Enter open · S save

Poisoning-Based Backdoors Are Not Cryptographic

Masahiro Kaneko

MBZUAI

Backdoor Attacks LLM Backdoor Attacks Backdoor Defense in LLMs

Open paper AskPDF arXiv

Defending against a backdoor requires recovering the verifier that decides whether an input is a trigger, since blocking individual triggers fails against an attacker who can generate new ones. If a verifier could be installed with the asymmetry of cryptographic authentication, where only the attacker can produce triggers and the defender cannot reconstruct the verifier, defense that learns the verifier from observations would be impossible in principle. We show that no such asymmetry can arise when the verifier enters training only through the labels of poisoned data, which covers the dirty-label backdoors that dominate the literature on language models. A training procedure that installs a verifier from poisoned data is itself a learner for that verifier. Hence any verifier that is hard to learn cannot be installed through poisoning, and any verifier that poisoning can install is reconstructible by a white-box defender holding as many labeled trigger observations as the attacker used. Hiding a secret key in the input or the weights does not change this. Experiments on instruction-tuned large language models confirm the dichotomy, with no configuration yielding a backdoor that is both effective for the attacker and irrecoverable for the defender.

Abstract

Backdoor Attacks

Explore similar work

Poisoning-Based Backdoors Are Not Cryptographic

When Stronger Triggers Backfire: A High-Dimensional Theory of Backdoor Attacks

Statistically Undetectable Backdoors in Deep Neural Networks