Organizations: School of Electronic Information, Huzhou College, Huzhou, 313000, China · School of Electronic Science and Engineering, South China Normal University, Foshan, 528225, China · School of Physics and Information Engineering, Guangdong University of Education, Guangzhou, 510303, China
Decision-based sparse attacks must reduce perturbation support using only the victim's predicted label. The best restoration strategy varies as support contracts: coarse removal works early, while tail states need finer, history-aware tests. We propose RLGPA, an online reinforcement learning method that schedules support-reduction executors and their granularity under a hard query budget. A masked Dueling Double DQN selects among seven stable policy--scale actions; each action produces support-relative restoration groups that are rebuilt after every macro decision. A feasibility-preserving executor commits only adversarial restorations and feeds support gain and query cost back to the planner. Under the matched top-1-label oracle, RLGPA reaches 61% targeted ASR at sparsity 0.01 on CIFAR-10 (vs. 25% SparseEvo, 0% Pointwise), and 75.5%/73.0% on ResNet50/ViT at sparsity 0.005 (vs. 29.0%/15.5%). A fixed-probe test reduces median L0 from 165 to 23 after 40 images without any update. State-blind and state-shuffled controls degrade significantly, confirming that the learned policy uses attack state; the main gains come from state-conditioned executor scheduling and dynamic repartition. The source code is publicly available at https://github.com/Acui001/RLGPA.git.
Figures & tables
Fig. 1 : Overview of RLGPA. The planner selects a policy–scale action from the current attack state, PSSE performs hard-label-verified support restoration, and the observed support reduction and query cost are returned for online updating.
Action
Executor
Ordering
κ
Coarse Primary
Primary
perturbation ascending
0.5
Base Primary
Primary
perturbation ascending
1.0
Coarse Safe
Safe
mean magnitude descending
0.5
Base Safe
Safe
mean magnitude descending
1.0
Fine Safe
Safe
mean magnitude descending
2.0
Base Memory
Memory
restoration memory descending
1.0
TABLE I : Canonical RLGPA macro-actions. Policy–scale semantics remain fixed when the current support is repartitioned.
TABLE II : Median sparsity and ASR at different query budgets. A comprehensive comparison among different attacks ( σ -zero, PGD 0 , Pointwise, SparseEvo, and RLGPA) on small and large scale balance datasets. The two white-box rows are reported at their converged distortion for reference only: their optimization cost is not a sequence of victim queries, so they are excluded from every query-efficiency comparison.
Fig. 2 : Comparison of RLGPA on CIFAR-10 (ResNet18) and ImageNet (ResNet50 and ViT). Rows correspond to dataset–model pairs, while columns present untargeted/targeted ASR–Sparsity and Query–Sparsity results. PGD 0 and σ -zero are white-box references included only in the ASR–Sparsity panels.
Setting
Query
Methods
ResNet50
ViT
Sparsity
0.001
0.005
0.001
0.005
Untargeted
σ -zero
100%
100%
100%
100%
PGD 0
74.0%
100.0%
46.5%
91.5%
2000
SparseEvo
21.0%
68.5%
27.0%
74.0%
RLGPA
76.5%
92.0%
32.5%
51.5%
5000
SparseEvo
21.5%
72.0%
30.0%
93.0%
TABLE III : ASR of ResNet50 and ViT under attacks at different query budgets and sparsity thresholds. A comprehensive comparison among different attacks ( σ -zero, PGD 0 , SparseEvo, and RLGPA) on small and large scale balanced evaluation sets from ImageNet.
Fig. 3 : Visual comparison of targeted attacks between RLGPA and SparseEvo on ImageNet. Query budgets: 100, 500, 1,000, 5,000, 10,000, and 20,000. For each method, the upper row shows the adversarial image and the lower row its perturbation map with normalized sparsity. Target-class reference (left) and source image (right).
Fig. 4 : Visual evolution of targeted adversarial examples generated by RLGPA on ViT and ResNet50 on ImageNet. Two source–target pairs are shown. Query budgets: 1,000, 5,000, 10,000, 15,000, and 20,000. Target-class reference (left), source image (right); enlarged crops highlight residual perturbations at the final budget.
TABLE IV : Frozen performance on a fixed, disjoint 20-image probe set. No probe-time parameter or replay update is allowed. Median, Mean, and P90 are for L0 .
TABLE V : Matched planner attribution on targeted CIFAR-10. The final column is the paired Wilcoxon test against Full DQN. Median, Mean, and P90 are for L0 .
State variant
ASR @.01
Median
Mean
P90
Full state
.59
24.5
35.56
67.0
No action history/effectiveness
.54
27.0
37.77
71.1
No recent progress/stall
.54
29.0
38.64
73.7
No phase/budget
.50
30.5
36.41
71.0
Minimal legal mask
.48
31.5
46.67
90.2
State-shuffled, five-seed mean
.482
32.8
45.35
89.16
TABLE VI: State ablation for the final relative planner. Median, Mean, and P90 are for L0 .
Reward
ASR @.01
Median
Mean
P90
Support gain/query
.56
24.0
34.46
68.0
Full shaped reward
.59
24.5
35.56
67.0
No stall bonus
.52
29.0
39.00
71.1
No tail/stall bonuses
.50
30.0
39.42
74.2
No phase weights
.49
31.0
40.29
72.2
No tail bonus
.47
31.0
44.00
71.3
TABLE VII: Reward ablation on targeted CIFAR-10. Median, Mean, and P90 are for L0 .
TABLE VIII : Relative granularity and executor ablation. Median, Mean, and P90 are for L0 .
Phase
Action
Support gain /query
Interpretation
Early
Coarse Primary
6.970
aggressive initial contraction
Middle
Coarse Primary
1.884
continued coarse removal
Late
Base Memory
0.459
history-guided tail refinement
TABLE X : Most productive action by support-search phase.
Fig. 5 : ImageNet targeted hyperparameter sensitivity on ResNet50. Curves aggregate three seed-specific median-sparsity trajectories over the same 100-image stream; shaded regions denote 95% confidence intervals. The horizontal line marks the sparsity threshold 0.005 used in the main targeted comparison. Lower is better.
A pixel-sparse adversary may change a few pixels of an image to any value. Robustness to such attacks is currently measured with an ensemble, sparse-AutoAttack (sAA), because its members fail in different places: sparse projected gradient descent (sPGD) trusts a first-order model that a full pixel change breaks, and sparse random search (Sparse-RS) ignores the gradient even where it is informative. We show that both attacks are local searches, each a proposal distribution plus an acceptance rule, and ask what such a search can learn from the gradient. A Taylor bound says what to measure, and measurements at the states the search visits give five rules. The sign of the gradient is its most reliable use at a single pixel and sets a new pixel's value. Its ranking of candidate pixels should be followed loosely, and its ranking of the pixels to remove, which is often more reliable, can be followed more closely. A pixel should not be proposed twice before a move is accepted, and acceptance should be decided by the loss alone. These rules define Gradient-guided Random Search (GRS), which runs once and costs one forward and one backward pass per step. On 13 CIFAR-10 configurations and 5 ImageNet models, GRS matches or beats the best single member of sAA and is within one point of sAA itself or stronger on 17 of 18 settings, and it is significantly stronger than sAA on 8. It uses 3 instead of up to 7 forward-pass units, and on CIFAR-10 it runs 3.7 to 5.0 times faster than the members of sAA run in full.
Deep neural networks achieve strong performance across many tasks but remain vulnerable to adversarial perturbations. Adversarial training (AT) is one of the most effective defenses, yet it suffers from high memory cost on large models and the persistent clean-robust accuracy trade-off. Recent work introduces parameter-efficient fine-tuning, such as LoRA, into AT, but this constraint induces a notable robustness gap relative to full-parameter AT. In this work, we revisit parameter-efficient adversarial fine-tuning from a parameter space perspective. Through gradient analysis, we find that adversarial optimization is highly concentrated on a small subset of parameters, which we call Robustness-Critical Parameters (RCPs), suggesting that robustness is encoded in a sparse subspace. Building on this observation, we propose Sparse Adversarial Fine-Tuning (SAFT), which identifies RCPs via adversarial gradient saliency and performs AT by updating only these parameters while freezing the rest. Across architectures and datasets, SAFT outperforms LoRA-based methods in both clean and robust accuracy with fewer trainable parameters, and approaches or surpasses full parameter AT in robustness while updating only about 5% of parameters.
Zhaoxin Wang, Zihang Ding, Handing Wang
School of Artificial Intelligence, Xidian University, Xi’an, China
While decision-based black-box adversarial attacks present a severe security threat, current methodologies suffer from fundamental limitations. Pixel-wise attacks frequently introduce unnatural, high-frequency visual artifacts, while latent-space frameworks are confined by the limited search space of low-dimensional manifolds and inherent reconstruction flaws. To resolve these limitations, we propose Latent Geometric Chords (LGC) for Query-Efficient Decision-Based Adversarial Attacks alongside a variant, LGC-H. At its core, LGC navigates decision boundaries by executing a curvature-aware geometric search within a compressed semantic manifold. To guarantee high visual fidelity and circumvent dimensionality bottlenecks, we introduce a Residual-based Adversarial Generation (RAG) mechanism. RAG isolates semantic perturbations as geometric chords and superimposes them directly onto the original source image. RAG substantially resolves baseline reconstruction flaws and effectively doubles the permissible search space dimensions. Experimental results demonstrate that LGC achieves robust cross-dataset transferability and substantially outperforms state-of-the-art baselines. Notably, our method, LGC, minimizes perturbation magnitudes while achieving state-of-the-art visual fidelity--with a Structural Similarity Index Measure (SSIM) exceeding 0.99 and a Learned Perceptual Image Patch Similarity (LPIPS) below 0.01 at 5000 queries--and sustaining high attack success rates under stringent perceptual constraints, successfully compromising adversarially trained robust models. The source code is available at: https://github.com/eihmuekhine/Latent-Geometric-Chords.
Ei Hmue Khine, Yao Li, Jiebao Sun +3
School of Mathematics, Harbin Institute of Technology, Harbin 150001, China