Margin-Drop Coordinates for Cross-Budget Robustness Evaluation

Yanliang Huang; Zhen Zhang; Peng Xie; Wenyuan Wu; Sitong Zhu; Zhuoqi Zeng; Amr Alanwar

cs.LGAug 20, 2026

Margin-Drop Coordinates for Cross-Budget Robustness Evaluation

Authors: Yanliang Huang, Zhen Zhang, Peng Xie, Wenyuan Wu, Sitong Zhu, Zhuoqi Zeng, Amr Alanwar

Organizations: School of Computation, Information and Technology, Technical University of Munich, Munich, Germany · Department of Statistics, Ludwig-Maximilians-Universität München, Munich, Germany · School of Engineering, Hainan Bielefeld University of Applied Sciences, Hainan, China

Abstract

Fixed-budget robustness evaluation can select the wrong frozen vision encoder. An encoder that survives a shallow attack may lose most of that robustness when the same evaluation is strengthened. We ask whether the shallow evaluation contains enough information to identify this budget fragility. For each clean-correct sample, the evaluation records the clean pairwise margin, the first-order linearized margin-drop scale, the margin drop from a clean-start one-step attack, and the drop reached by an iterative attack. Normalizing by that scale gives three margin-drop coordinates capturing clean margin slack, one-step shortfall, and drift, where drift is the additional normalized margin drop the iterative attack reaches beyond the one-step perturbation. Together, they reconstruct the normalized post-attack margin and therefore the pass-or-fail outcome. Across 42 pretrained frozen vision encoders, the shallow survival rate carries essentially no rank information about subsequent PGD-10 to PGD-200 collapse, at Spearman -0.006, while the median shallow drift coordinate ranks the same collapse at +0.811. The result persists in a held-out encoder pool and under an $\ell_\infty$ evaluation. With deep evaluation limited to 11 encoders, ranking by shallow drift recovers 11 of the 17 high-collapse encoders, compared with 5 under survival-rate ranking. The full coordinate decomposition further distinguishes cases that share the same fixed-budget residual but diverge at deeper budgets, and separates margin repair from drift repair under interventions, revealing distinct repair paths that endpoint robustness alone does not identify.

Figures & tables

Panel of Figure 1 — Figure 1: Coordinate decomposition of measured margin drop. (Left) Normalized fixed-competitor pairwise margin over a two-dimensional slice of the $\ell_{2}$ threat set for a real ViT-B/16 sample. The two in-plane axes are signed input-space coordinates, with $\alpha$ along the first-order attack direction at the clean point and $\beta$ along the component of the selected PGD perturbation orthogonal to it, while the surface height is the normalized margin. Because the two directions are orthonormal, the dashed circle $\alpha^{2}+\beta^{2}=\varepsilon^{2}$ , with $\varepsilon=0.25$ , is the threat-set boundary. Blue, orange, and red mark the clean input, the one-step perturbation, and the selected iterative perturbation. The coordinate $\kappa$ is the normalized shortfall of the realized one-step drop relative to the first-order prediction, while drift $\rho$ is the additional normalized drop reached by the iterative attack beyond the one-step perturbation. The left panel shows the geometry orthogonal to the first-order attack direction associated with this drift. (Right) The same recorded quantities give the clean margin slack $\lambda$ and reconstruct the normalized post-attack margin $m=\lambda+\kappa-1-\rho$ , so the sample survives the stated attack when $m>0$ .

Figure 2: Shallow-budget drift ranks deeper-budget collapse. (Left) Across the 42-encoder

\ell_{2}

evaluation,

\bar{\rho}_{10}

ranks

\Delta R(\mathrm{PGD}\text{-}10\times 5\to\mathrm{PGD}\text{-}200\times 5)

at Spearman

+0.811

. (Right) The shallow survival rate

R_{10}

has essentially no rank relationship with the same collapse target.

Check	Target	Result
Primary evaluation	$\Delta R(\mathrm{PGD}\text{-}10\times 5\to\mathrm{PGD}\text{-}200\times 5)$	$+0.811$
One-restart evaluation	$\Delta R(\mathrm{PGD}\text{-}10\times 1\to\mathrm{PGD}\text{-}200\times 5)$	$+0.717$
Conditional collapse	$\Pr(\mathrm{fail}_{200}\mid\mathrm{pass}_{10})$	$+0.669$
Family-cluster bootstrap	95% CI for primary Spearman	$[+0.423,+0.945]$
Dynamic all-class margin	$\Delta R_{\mathrm{all}}(\mathrm{PGD}\text{-}10\times 5\to\mathrm{PGD}\text{-}200\times 5)$	$+0.797$
Cross-entropy Auto-PGD top- $1$	$\Delta R_{\mathrm{top1}}^{\mathrm{CE/APGD}}(10\to 100)$	$+0.833$

Table 1: Validation of shallow drift ranking. Correlation rows report Spearman statistics. The family-cluster row reports a bootstrap interval for the primary correlation.

Panel of Figure 3 — Figure 3: Residual compression hides distinct shortfall–drift structure. (Left) Encoder-level coordinates on the $n=40$ analytical subset show that similar net residuals can correspond to different drift and cross-budget collapse. (Right) In a DeiT3 ViT-B/16 band matched on $\tau_{10}$ and $\lambda$ , flat and cancellation samples have sharply different $\mathrm{PGD}\text{-}200$ survival.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

		Deep-evaluation capacity $c$
Threshold	$\|\mathcal{F}\|$	$5$	$8$	$11$	$15$	$21$
$10$ pp	$29$	$5$ / $3$ / $3.5$	$8$ / $5$ / $5.5$	$11$ / $7$ / $7.6$	$15$ / $10$ / $10.4$	$21$ / $16$ / $14.5$
$15$ pp	$24$	$5$ / $3$ / $2.9$	$8$ / $4$ / $4.6$	$11$ / $6$ / $6.3$	$15$ / $9$ / $8.6$	$21$ / $13$ / $12.0$
$20$ pp	$17$	$5$ / $2$ / $2.0$	$8$ / $3$ / $3.2$	$11$ / $5$ / $4.5$	$12$ / $6$ / $6.1$	$15$ / $9$ / $8.5$
$25$ pp	$14$	$5$ / $1$ / $1.7$	$8$ / $2$ / $2.7$	$10$ / $3$ / $3.7$	$11$ / $4$ / $5.0$	$13$ / $7$ / $7.0$
$30$ pp	$12$	$4$ / $0$ / $1.4$	$7$ / $1$ / $2.3$	$9$ / $2$ / $3.1$	$10$ / $3$ / $4.3$	$11$ / $6$ / $6.0$

Appendix

Table 8: Selection sensitivity against the

\mathrm{PGD}\text{-}10\times 5\to\mathrm{PGD}\text{-}200\times 5

collapse target on the

42

-encoder pool. Each cell reports drift ranking, survival-rate ranking, and the uniform random expectation, in that order. The threshold row

20

pp and capacity column

11

is the operating point reported in Section 4.3 .

Figure 4: Marginal and partial rank statistics on the analytical

n=40

subset. Points show Spearman or partial-Spearman correlations with

\Delta R(\mathrm{PGD}\text{-}10\times 5\to\mathrm{PGD}\text{-}200\times 5)

. Bars show 95% bootstrap confidence intervals. Rows of the form

a\mid b

report the partial rank correlation for

a

after controlling for

b

. The same values are tabulated in Table 11 .

Figure 5: Perturbation-vector geometry analysis on six representative encoders and

600

clean-correct samples. (Top) Selected PGD perturbations are projected into components parallel and perpendicular to the initial gradient ray. (Bottom) Decomposing the PGD-over-one-step margin drop by

\rho_{10}

tertiles shows that the off-ray component accounts for most of the increase, while the on-ray projection component can be negative because the projection is not the on-ray maximizer.

Figure 6: Per-sample flip rate for the score

\rho_{10}-m_{10}

among

\mathrm{PGD}\text{-}10

survivors. Points show observed pass-

10

to fail-

200

flip rates within equal-count score bins, plotted against the bin-median score. Error bars show 95% binomial confidence intervals. The score reaches pooled AUC

0.824

for predicting pass-

10

to fail-

200

flips.

Figure 7: Coordinates separate drift repair from survival-margin repair. (Top) On ViT-B/16, both interventions improve pairwise robust retention across

\mathrm{PGD}\text{-}10\times 5

\mathrm{PGD}\text{-}50\times 10

, and

\mathrm{PGD}\text{-}200\times 5

, with stronger deep-budget retention for Coord. (Bottom) Component-median movements show that larger drift reduction can yield a smaller survival-margin gain when margin slack is eroded. Dashed diagonals are visual guides for combined slack-and-drift movement. Exact

\Delta\bar{m}

values are computed per sample before aggregation, with

\Delta\bar{\kappa}

outside this two-coordinate plane. Labeled

\Delta\bar{m}

values are median per-sample changes.

Adversarial Robustness

Adversarial robustness studies how well AI systems withstand deliberately crafted inputs, training-data corruption, and other attacks.

See all 484 papers

Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation
Jul 22, 2026Luca Scionis, Luca Melis, Maura Pintor +5Adversarial Attacks Neural Network Robustness
Sapienza University of Rome, Italy · University of Cagliari, Italy · University of Genoa, Italy
What Paired Evaluations Reveal under Visual Perturbations
Sep 28, 2026Yongda Wei, Chen Zhang, Yifei Wang +4Image Corruption Robustness Adversarial Robustness
Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?
Apr 2, 2026Kamalasankari Subramaniakuppusamy, Jugal GajjarExplainability Evaluation Perturbation-Based Feature Attribution
The George Washington University Washington D.C.

Jul 22, 2026J/K move · Enter open · S save

Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation

Luca Scionis, Luca Melis, Maura Pintor, Fabio Brau+4

Sapienza University of Rome, Italy · University of Cagliari, Italy · University of Genoa, Italy

Adversarial Attacks Neural Network Robustness Adversarial Robustness

Open paper AskPDF arXiv

Adversarial robustness is commonly evaluated with predefined attack ensembles, such as AutoAttack, at a single perturbation budget

\varepsilon

and on a selective choice of perturbation norms. We argue this formulation is fundamentally limited. First, robustness--perturbation curves may intersect or decay at different rates across models, making single-

\varepsilon

rankings unstable. Second, current ensembles provide no evidence of optimality, leaving an unknown gap to worst-case performance. Third, fixed attack configurations provide no systematic control over the trade-off between attack strength and evaluation cost. To address these limitations, we introduce a unified evaluation framework based on a comprehensive pool of minimum-norm attacks and robustness--perturbation curves across

\ell_0

\ell_1

\ell_2

and

\ell_\infty

norms. We define the attack frontier as the worst-case robustness estimate the attack pool produces against a model. We then formalize evaluation as a frontier-approximation problem, constructing minimum-norm attack ensembles, optimized subsets of the comprehensive pool, that approach the frontier under a controllable query budget, with larger budgets monotonically tightening the estimate. Furthermore, we define the defense frontier as the maximum robustness across the model set at each perturbation size. We finally propose the Defense Optimality Index to rank defenses by their gap to the defense frontier, providing a ranking without selecting a reference

\varepsilon

. On CIFAR-10 and ImageNet, our ensembles match or exceed AutoAttack on most defenses at every budget tier, at fixed and controllable query cost, offering practitioners a query-controlled, curve-based alternative to fixed-

\varepsilon

evaluation.

Abstract

Adversarial Robustness

Explore similar work

Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation

What Paired Evaluations Reveal under Visual Perturbations

Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?