Score-based black-box adversarial attacks pose a serious threat to deployed machine learning systems by exploiting repeated model queries to craft adversarial examples, requiring neither access to model parameters nor architecture. Existing defenses predominantly rely on stochastic output perturbations, which fail to prevent determined attackers from recovering stable gradient estimates by increasing query budgets or averaging over randomness. In this work, we propose the Gradient Confusion Defense (GCD), a retraining-free, deterministic test-time wrapper that actively shapes the probability vector returned to querying adversaries. By solving a constrained optimization problem grounded in the model's Jacobian, GCD aligns the exposed logit-gradient with a target confusion direction that is maximally misaligned with the true adversarial loss gradient---increasing the query complexity of attacks without sacrificing predictive utility or top-1 accuracy. We provide rigorous theoretical analysis demonstrating that GCD provably slows gradient-estimation-based attacks and induces systematic accept/reject errors in random search-based attacks, with the confusion effect amplifying as the fraction of defended queries increases. Extensive experiments on CIFAR-10 and ImageNet across multiple architectures (ResNet-18, ResNet-50, ViT-B/16) and four state-of-the-art attacks confirm these predictions: GCD achieves state-of-the-art robustness, while incurring zero clean accuracy degradation. %Crucially, unlike randomization-based defenses, GCD's deterministic structure provides resilience against adaptive attacks that exploit stochasticity, making it well-suited for security-critical deployment settings.
Figures & tables
Model
Defense
Acc. Drop
Attacks (Success % / Avg. Queries)
Bandit- ℓ2
Bandit- ℓ∞
Square- ℓ2
Square- ℓ∞
Sign- ℓ∞
BrusLe- ℓ0
ResNet-18
None
-
82.68% / 545.68
98.03% / 186.16
87.76% / 466.29
99.38% / 112.46
98.97% / 230.05
79.78% / 234.82
iRND
1%
11.63% / 333.97
41.47% / 186.49
24.92% / 290.21
80.52% / 178.96
66.49% / 233.48
82.9% / 220.76
oRND
0.6%
12.53% / 354.43
47.70% / 158.43
20.98% / 188.31
64.09% / 113.92
38.83% / 170.05
68.6% / 201.10
RLS
0%
04.15% / 345.9
25.11% / 23.96
12.76% / 268.69
47.31% / 90.88
21.06% / 111.41
51.77% / 316.57
AAA
0.1%
03.33% / 318
38.08% / 132.26
9.77% / 174.91
43.41% / 82.04
32.09% / 177.10
66.78% / 290.17
Table 1: Black-box attack results on CIFAR-10. ASR and average number of queries are reported.
Model
Defense
Acc. Drop
Attacks (Success % / Avg. Queries)
Bandit- ℓ2
Bandit- ℓ∞
Square- ℓ2
Square- ℓ∞
Sign- ℓ∞
ResNet-50
None
-
46.73% / 761.22
87.05% / 251.79
81.45% / 483.65
99.47% / 133.90
91.33% / 328.71
iRND
0.2%
11.02% / 440.30
45.56% / 141.79
26.66% / 214.17
83.49% / 126.10
60.78% / 229.78
oRND
1.7%
51.47% / 229.94
75.95% / 103.83
59.84% / 208.49
83.52% / 96.91
69.47% / 182.20
RLS
0%
6.68% / 618.42
47.80% / 143.72
16.29% / 143.03
59.82% / 104.07
21.37% / 109.13
AAA
0.3%
11.80% / 356.96
37.94% / 85.10
15.69% / 32.05
47.06% / 33.59
23.6% / 109.37
Table 2: Black-box attack results on ImageNet. ASR and average number of queries are reported.
Defense
Min ℓ1
Avg ℓ1
Max ℓ1
iRND
0.00
0.11
1.03
oRND
0.00
0.26
1.77
RLS
0.00
0.33
1.74
AAA
0.00
0.28
1.60
GCD
0.07
0.58
1.25
Table 3: ℓ1 distance between the original output of the model y^ and defense output y~ for 1000 ImageNet samples.
Figure 1: Polar histograms of gradient direction misalignment. For 10 randomly selected samples from the ImageNet test set, we plot the angular deviation θ=∠(∇xf(x),∇xf~(x)) between the direction ∇xf~(x) induced by the defended score responses (i.e., the direction the attacker would follow) and the true loss gradient ∇xf(x) . Each color corresponds to 200 square- ℓ∞ attack iterations (200 queries used to generate adversarial example for a sample). Dashed rays and numeric labels indicate the mean deviation for each sample.
Figure 2: Angular deviation between defense-exposed and undefended gradient directions over 100 Square Attack iterations on 10 CIFAR-10 samples.
Defense
EOT
Pen-Square
Max(EOT,Pen)
AAA
21.94% / 85.44
40.53% / 306.08
40.53%
RLS
36.19% / 215.83
23.55% / 85.44
36.19%
GCD
24.04% / 159.69
30.12% / 209.33
30.12%
Table 4: Adaptive Attacks on GCD, RLS, and AAA. We incorporate two adaptive techniques—Expectation over Transformation (EOT) and Pendulum—into the Square Attack framework. For both adaptive variants, we set k=10 and evaluate performance on 1000 ImageNet samples using ViT-B/16.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Symbol
Description
General Model and Inputs
X
Input space, X⊆[0,1]d
d
Dimension of the input space
[K]
Label set {1,…,K}
M
Classifier mapping X→ΔK
z(x)
Logits vector output by the model for input x
Appendix
Table 5: Summary of Notations and Symbols used in the Gradient Confusion Defense (GCD) paper.
Black-box adversarial attacks that minimize only the ground-truth confidence suffer from class drift: perturbations wander through the feature space without committing to a specific adversarial class, wasting queries on diffuse, undirected progress. We introduce Opportunistic Target Selection (OTS), a lightweight wrapper that switches an untargeted attack to a targeted objective early in its trajectory, locking onto whichever non-true class currently leads. OTS requires no architectural modification to the underlying attack, no gradient access, and no a priori target-class knowledge. We validate OTS on three score-based attacks (SimBA, Square Attack with cross-entropy loss, and Bandits) across five standard ImageNet classifiers (4,500 runs). On random-search attacks, OTS closely tracks oracle performance, with gains up to +27 pp in success rate and 43% relative reduction in censored-mean iterations on ResNet-50. On gradient-estimation attacks (Bandits) and attacks with margin loss, OTS is redundant, a negative result that reinforces our interpretation of OTS as a margin-loss surrogate. On adversarially-trained models, a bimodal difficulty distribution eliminates the regime where targeting helps.
Machine learning models are increasingly adapted in various domains. However, adversarial examples pose a significant threat to the reliable deployment of these models. In recent years, some powerful adversarial example attacks have been proposed for the fast and query-efficient generation of adversarial examples, even in black-box scenarios, highlighting the need for scalable, low-cost, and powerful defenses. In this work, we present two contributions to the domain of black-box adversarial example attacks and defenses. First, we propose Random Logit Scaling (RLS), a randomization-based defense against black-box score-based adversarial example attacks. RLS is a plug-and-play, post-processing defense that can be implemented on top of any existing ML model with minimal effort. The idea behind RLS is to confuse an attacker by outputting falsified scores resulting from randomly scaled logits while maintaining the model accuracy. We show that RLS significantly reduces the success rate of state-of-the-art black-box score-based attacks while preserving the accuracy and minimizing confidence score distortion compared to state-of-the-art randomization-based defenses. Second, we introduce a novel adaptive attack against AAA, a SOTA non-randomized black-box defense against black-box score-based attacks that also modifies output logits to confuse attackers, demonstrating its vulnerability against adaptive attacks.
Hamid Dashtbani, Mehdi Dousti Gandomani, AmirMahdi Sadeghzadeh
Department of Computer Engineering Sharif University of Technology
This work studies the robust evaluation of iterative stochastic purification defenses under white-box adversarial attacks. Our key technical insight is that gradient checkpointing makes exact end-to-end gradient computation through long purification trajectories practical by trading additional recomputation for substantially lower memory usage. This enables full-gradient adaptive attacks against diffusion- and Langevin-based purification defenses, where prior evaluations often resort to approximate backpropagation due to memory constraints. These approximations can weaken the attack signal and risk overestimating robustness. In parallel, stochasticity in iterative purification is frequently under-controlled, even though different purification trajectories can substantially change reported robustness metrics. Building on this insight, we introduce a memory-efficient full-gradient evaluation framework for stochastic purification defenses. The framework combines checkpointed backpropagation with evaluation protocols that control stochastic variability, thereby reducing memory bottlenecks while preserving exact gradients. We evaluate diffusion-based purification and Langevin sampling with Energy-Based Models (EBMs), demonstrating that full-gradient attacks uncover vulnerabilities missed by approximate-gradient evaluations. Our framework yields stronger state-of-the-art ℓ∞ and ℓ2 white-box attacks and further supports probing out-of-distribution robustness. Overall, our results show that exact-gradient evaluation is essential for reliable benchmarking of iterative stochastic defenses.
Yuan Du, Mitchel Hill, HanQin Cai
Department of Statistics and Data Science University of Central Florida Orlando, FL 32816, USA · InnoPeak Technology Palo Alto, CA 94303, USA · Department of Statistics and Data Science Department of Computer Science University of Central Florida Orlando, FL 32816, USA