Adversarial Machine Learning research is currently centered on the search for the epsilon-ball: minimal perturbations designed to fool models while remaining imperceptible to humans. We argue that this fixation on ℓp norms is a metrological trap that communicates a distorted perception of risk to defenders, one that is unaligned with real-world system vulnerabilities. This paradigm misleads defenders into optimizing for narrow, mathematical robustness at the expense of systemic security, while simultaneously incentivizing attackers to exploit non-overlapping alternative adversarial pathways. Ultimately, we argue that the dominance of the ℓp ball does not just fail to reduce harm---it may actively facilitate it by communicating a false sense of security against the wrong threats.
Figures & tables
Category
ϵ
Pois.
Jail.
Sem.
Patch
Def
Audio
11
18
25
9
2
1
General
293
118
98
114
13
28
Graph
27
30
20
14
5
9
Image
334
153
143
110
31
51
Multim.
60
38
43
23
4
5
RL
46
27
51
43
11
20
Table 1: Threat frequency from 4311 Adversarial ML papers from ICML/ICLR/NeurIPS over 2022–2025, categorized by primary threat model (all entries are distinct) following Appendix D . Abbreviations: ϵ : small metric perturbations, Pois.: Poisoning, Jail.: Jailbreak, Sem.: Semantic, Def.: Defense. Note: the majority of Poisoning, Jailbreaking, Patch, and Defense papers employ an ϵ threat model (see Section 2 and Appendix D ).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Conference
2023
2024
2025
Total
ICLR
605
316
498
1419
ICML
251
341
470
1062
NeurIPS
453
569
808
1830
Total
1309
1226
1776
4311
Appendix
Table 2: Distribution of Analyzed Papers by Venue and Year.
Mathematical Constraint
Count
Unconstrained Content (Dirty Label)
270
Spatially Constrained (L0 / Mask)
202
Semantically Constrained (Clean Label)
134
Magnitude Constrained (Lp-Norm)
112
Appendix
Table 3: Data Poisoning by Mathematical Constraint Type.
Mathematical Constraint
Count
Distributional (Wasserstein/EMD)
220
General Semantic / Unspecified
184
Discrete Distance (Levenshtein)
26
Geometric (Rotation/Spatial)
7
Appendix
Table 4: Breakdown of Semantic/Transformation Constraints.
Adversarial robustness evaluations of large language models (LLMs) typically report attack success rate (ASR) under fixed query budgets, implicitly treating all attacks as equally costly. In practice, the computational expense of different attack strategies can vary by orders of magnitude. Consequently, ASR at a fixed budget can obscure the true effort required to jailbreak a model, thereby making it hard to determine whether an attack's cost justifies its payoff to the attacker. We propose a compute-aware evaluation framework based on computational pressure, measured in cumulative floating-point operations (FLOPs), as a proxy for adversarial effort. We introduce risk-compute curves, which map compute budgets to attack risk, and derive two metrics that summarize the average pressure required for a given attack to succeed. Across ten models spanning three families and four different stages in language model training and alignment, evaluated with three attack strategies (gradient-based, iterative refinement, and template-based) on two jailbreak robustness benchmarks, we find: (1) alignment training has non-monotonic effects on compute-space robustness; (2) scaling model size reduces gradient-based attack effectiveness but has limited impact on cheaper template-based attacks; (3) gradient-based attacks optimized on a surrogate model can transfer to a separate target model, providing a way to reduce attacker costs; (4) compute cost varies by up to ≈5× across harm categories within a single model; and (5) safety-aligned RL increases aggregate cost while leaving some categories disproportionately accessible. We release our framework to enable compute-aware risk assessment and evaluation.
Beneath the apparent diversity of foundation models lies an increasingly homogeneous technical stack of common training pipelines, distillation methods, and data. These shared algorithmic choices can expose an otherwise distinct model ecosystem to concentrated risks leading to a vulnerability monoculture. Yet current practice in assessing adversarial robustness is restricted to individual robustness scores and attack transfer rates, leaving open the unexplored fundamental question: how many attacks suffice for catastrophic widespread compromise? In this paper, we investigate the empirical nature of shared vulnerabilities within a model pool and introduce Systemic Adversarial Robustness (SAR), a novel measure of shared vulnerability of a model pool through the attack diversity needed to exploit it. Specifically, SAR interpolates between single-attack transferability and independent per-model robustness: small values reveal a vulnerability monoculture, whereas large values indicate that failures are dispersed across models. We theoretically formulate SAR as a partial set-cover problem with provable approximation guarantees---enabling the development of scalable algorithms to empirically measure the systemic adversarial robustness. We empirically test the shared vulnerability of vision and large language models and characterize their shared collective vulnerability curve. Specifically, we demonstrate that model pools distilled from DeepSeek-R1 and Claude Opus 4.6 exhibit lower estimated systemic robustness than their corresponding parent pools, revealing a potential systemic cost of frontier-model distillation. Our framework seeks to establish vulnerability monoculture as a measurable target for a more wholistic adversarial evaluation of model populations.
Leo Elmecker-Plakolm, Lukasz Szpruch, Avishek Joey Bose +1
Imperial College London · The University of Edinburgh · The Alan Turing Institute +1
The vulnerability of ML models to adversarial examples has recently emerged as a major concern. While adversarial training is one of the most effective countermeasures to this issue, its high computational cost remains an obstacle to practical deployment. Recent progress in reducing this cost has relied, in the case of linear models, on a formal equivalence between the adversarial risk and a simpler form of regularized risk. This enabled significantly more efficient training procedures, which naturally raises the question of whether such an equivalence can be extended beyond linear models. In this work, we formally show that no such equivalence is possible for two-layer networks. Our proofs proceed via a reduction to key properties that fundamentally separate the adversarial risk from any simple regularized risk which would only exhibit a weak form of data dependence. Beyond this setting, we provide empirical evidence on Wide-ResNets indicating that the same type of impossibility persists in deeper and more expressive architectures.
David A. R. Robin, Rafael Pinot, Yann Chevaleyre
LAMSADE, Dauphine · Universit´e Paris Dauphine PSL Research University · LPSM, Jussieu +1