cs.CVNov 17, 2026

Propose with the Gradient, Accept with the Loss: One Hedged Attack for Pixel-Sparse Robustness

Authors: Hon Ming Yan, Chun Pong Lau

Organizations: City University of Hong Kong

Abstract

A pixel-sparse adversary may change a few pixels of an image to any value. Robustness to such attacks is currently measured with an ensemble, sparse-AutoAttack (sAA), because its members fail in different places: sparse projected gradient descent (sPGD) trusts a first-order model that a full pixel change breaks, and sparse random search (Sparse-RS) ignores the gradient even where it is informative. We show that both attacks are local searches, each a proposal distribution plus an acceptance rule, and ask what such a search can learn from the gradient. A Taylor bound says what to measure, and measurements at the states the search visits give five rules. The sign of the gradient is its most reliable use at a single pixel and sets a new pixel's value. Its ranking of candidate pixels should be followed loosely, and its ranking of the pixels to remove, which is often more reliable, can be followed more closely. A pixel should not be proposed twice before a move is accepted, and acceptance should be decided by the loss alone. These rules define Gradient-guided Random Search (GRS), which runs once and costs one forward and one backward pass per step. On 13 CIFAR-10 configurations and 5 ImageNet models, GRS matches or beats the best single member of sAA and is within one point of sAA itself or stronger on 17 of 18 settings, and it is significantly stronger than sAA on 8. It uses 3 instead of up to 7 forward-pass units, and on CIFAR-10 it runs 3.7 to 5.0 times faster than the members of sAA run in full.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. RLGPA: A Reinforcement Learning-Guided Granularity Planning Attack for Decision-Based Sparse Adversarial Examples

    Nov 5, 2026Chao Zhou, Shenghua He, Jiaxin Mo +2Adversarial AttacksBlack-Box Adversarial Attacks

  2. SDM: A Powerful Tool for Evaluating Model Robustness

    May 19, 2026Xinlei Liu, Tao Hu, Jichao Xie +3Adversarial AttacksNeural Network Robustness

  3. ExGrad: Beyond Suppression - Explicit Clean-Gradient Subtraction in Adversarial Training

    Oct 21, 2026Zhiyuan Ye, Xiangyu Zhou, Ji Qi +2Adversarial TrainingAdversarial Robustness