Dummy-Aware Weighted Attack (DAWA): Breaking the Safe Sink in Dummy Class Defenses

Yunrui Yu; Xuxiang Feng; Pengda Qin; Pengyang Wang; Kafeng Wang; Cheng-zhong Xu; Hang Su; Jun Zhu

cs.LGMar 31, 2026

Dummy-Aware Weighted Attack (DAWA): Breaking the Safe Sink in Dummy Class Defenses

Authors: Yunrui Yu, Xuxiang Feng, Pengda Qin, Pengyang Wang, Kafeng Wang, Cheng-zhong Xu, Hang Su, Jun Zhu

Organizations: Tsinghua University · University of Macau

Abstract

Adversarial robustness evaluation faces a critical challenge as new defense paradigms emerge that can exploit limitations in existing assessment methods. This paper reveals that Dummy Classes-based defenses, which introduce an additional "dummy" class as a safety sink for adversarial examples, achieve significantly overestimated robustness under conventional evaluation strategies like AutoAttack. The fundamental limitation stems from these attacks' singular focus on misleading the true class label, which aligns perfectly with the defense mechanism--successful attacks are simply captured by the dummy class. To address this gap, we propose Dummy-Aware Weighted Attack (DAWA), a novel evaluation method that simultaneously targets both the true label and dummy label with adaptive weighting during adversarial example synthesis. Extensive experiments demonstrate that DAWA effectively breaks this defense paradigm, reducing the measured robustness of a leading Dummy Classes-based defense from 58.61% to 29.52% on CIFAR-10 under l_infty perturbation (epsilon=8/255). Our work provides a more reliable benchmark for evaluating this emerging class of defenses and highlights the need for continuous evolution of robustness assessment methodologies.

Figures & tables

Figure 1: Attack effectiveness comparison on CIFAR-10 with

\ell_{\infty}

constraint under PGD-AT+DUCAT defense [ 28 ] . Our DAWA (100 iterations, non-targeted) and DAWA mt (1,000 iterations, combined non-targeted/targeted) achieve

35.60\%

and

26.42\%

robust accuracy respectively, outperforming PGD (

60.64\%

), C&W (

71.72\%

), MIFPE (

63.10\%

), and AutoAttack (

56.80\%

Figure 2: Convergence speed comparison of four attacks (PGD, C&W, MIFPE, and DAWA) against PGD-AT+DUCAT defense [ 28 ] on CIFAR-10. The gray dashed line represents the AutoAttack robust accuracy (58.61%). The number annotation on the DAWA curve indicates the iteration (3) at which DAWA first surpasses AutoAttack’s performance, demonstrating its faster convergence.

Figure 3: Ablation study on the hyperparameter

c

(controlling

\alpha

smoothing in the loss function) within the DAWA evaluation strategy. The plot illustrates the untargeted attack robustness (measured after 100 iterations) as a function of

\log_{10}(c)

ranging from

10^{-1.0}

10^{2.0}

, for models trained on CIFAR-10 with ResNet-18 under three strategies: PGD-AT + DUCAT, MART + DUCAT, and Consistency + DUCAT.

Adversarial Robustness

Adversarial robustness studies how well AI systems withstand deliberately crafted inputs, training-data corruption, and other attacks.

See all 479 papers

Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation
Jul 22, 2026Luca Scionis, Luca Melis, Maura Pintor +5Adversarial Attacks Neural Network Robustness
Sapienza University of Rome, Italy · University of Cagliari, Italy · University of Genoa, Italy
Wasserstein Robust Classification Error: Surrogate Loss Misalignment and Distributional Attacks
Oct 11, 2025Bach C. Le, Tung V. Dao, Binh T. Nguyen +1Robust Classification Adversarial Robustness
College of Engineering & Computer Science, VinUniversity, Hanoi, Vietnam
Sensitivity as a Double-Edged Sword: A Trade-off Between Discriminability and Adversarial Robustness
Jun 1, 2026Kai WangPrototype-Based Classification Neural Network Robustness

Jul 22, 2026J/K move · Enter open · S save

Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation

Luca Scionis, Luca Melis, Maura Pintor, Fabio Brau+4

Sapienza University of Rome, Italy · University of Cagliari, Italy · University of Genoa, Italy

Adversarial Attacks Neural Network Robustness Adversarial Robustness

Open paper AskPDF arXiv

Adversarial robustness is commonly evaluated with predefined attack ensembles, such as AutoAttack, at a single perturbation budget

\varepsilon

and on a selective choice of perturbation norms. We argue this formulation is fundamentally limited. First, robustness--perturbation curves may intersect or decay at different rates across models, making single-

\varepsilon

rankings unstable. Second, current ensembles provide no evidence of optimality, leaving an unknown gap to worst-case performance. Third, fixed attack configurations provide no systematic control over the trade-off between attack strength and evaluation cost. To address these limitations, we introduce a unified evaluation framework based on a comprehensive pool of minimum-norm attacks and robustness--perturbation curves across

\ell_0

\ell_1

\ell_2

and

\ell_\infty

norms. We define the attack frontier as the worst-case robustness estimate the attack pool produces against a model. We then formalize evaluation as a frontier-approximation problem, constructing minimum-norm attack ensembles, optimized subsets of the comprehensive pool, that approach the frontier under a controllable query budget, with larger budgets monotonically tightening the estimate. Furthermore, we define the defense frontier as the maximum robustness across the model set at each perturbation size. We finally propose the Defense Optimality Index to rank defenses by their gap to the defense frontier, providing a ranking without selecting a reference

\varepsilon

. On CIFAR-10 and ImageNet, our ensembles match or exceed AutoAttack on most defenses at every budget tier, at fixed and controllable query cost, offering practitioners a query-controlled, curve-based alternative to fixed-

\varepsilon

evaluation.

Abstract

Adversarial Robustness

Explore similar work

Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation

Wasserstein Robust Classification Error: Surrogate Loss Misalignment and Distributional Attacks

Sensitivity as a Double-Edged Sword: A Trade-off Between Discriminability and Adversarial Robustness