Frequency-based adversarial attacks have recently grown popular by exploiting spectral sensitivities shared across neural architectures. Unlike spatial perturbations, frequency-based attacks expose deeper vulnerabilities, making them especially valuable for robust evaluation of safety-critical and security-sensitive applications. Yet, existing approaches are typically not derived as solutions to an optimization problem that explicitly captures transform-domain structure. In this paper, we propose a methodology for crafting principled frequency-based adversarial attacks, via a dedicated optimization framework. A cornerstone of our method hinges on the introduction of a perturbation constraint set, tied to highly structured non-orthogonal transforms, well-known for their flexible, non-predefined frequency handling. We prove that the attacks emerge as weighted ℓ2-projections onto this set, yielding a general and controlled attack generation mechanism. By this, we provide a clear geometric attack characterization, ensuring alignment between the optimization objective and the perturbation constraint. We assess our framework on standardized datasets, for pretrained and adversarially robust models. Results highlight that our attacks, being solutions to an optimization problem, over a structured perturbation set, are highly effective, even across different, unseen architectures. Our methodology could serve as a theoretical baseline for designing and analyzing transformed-based attacks, targeting fundamental model vulnerabilities, instead of mere architecture-specific artifacts typically studied in the robustness literature.
Figures & tables
Fig. 1 : Evaluation of the baselines ( ε=8/255 ) and our attack ( εDGF=η8/255 ), on an exemplary RobustBench model [ 1 ] .
Fig. 2 : Evaluation of the baselines ( ε=16/255 ) and our attack ( εDGF=ηε ), on an exemplary RobustBench model [ 9 ] .
ImageNet
CIFAR100
Attack
Pretrained
RobustBench
Pretrained
RobustBench
Proposed (spatial-frequency-based)
74.5
63.4
96.5
85.1
AutoAttack (spatial-based)
33.6
30.9
90.1
44.4
F-PGD (frequency-based)
32.8
32.1
74.2
46.0
Table 1: Average ASR (in %) across source-target model pairs, computed from the off-diagonal entries of the heatmaps of Fig. 3 . Bold letters indicate better cross-model transferability . Results support our methodological contribution: attacks derived from a principled optimization framework, based on structured overcomplete transforms that align more naturally with data geometry, can exploit shared model vulnerabilities more effectively than purely spatial-/frequency-based methods.
Fig. 3 : ASR (in % ) for both datasets, for all attacks, evaluated on different models. Color intensity indicates higher transferability . Each row represents a different source model, based on which an attack is created. We observe that our proposed attack enjoys a high transferability across a variety of models, and outperforms the baselines, thereby highlighting the importance of deriving attacks from a principled methodology.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 4 : Examples of images and their adversarial counterparts, with SSIM quality and ASR attack efficiency values, evaluated on a pretrained version of RegNet for ImageNet (top) and on a pretrained version of ResNet for CIFAR100 (bottom) , for ε=8/255 and ε=16/255 , respectively, on the baselines, and εDGF=η8/255εDGF=η16/255 , respectively, on our attack. Bold letters indicate the method with the best performance in terms of both evaluation metrics.
ImageNet
CIFAR100
Attack
Pretrained
RobustBench
Pretrained
RobustBench
Proposed (spatial-frequency-based)
75.4
63.5
96.9
83.8
AutoAttack (spatial-based)
44.0
41.6
89.6
64.9
F-PGD (frequency-based)
42.7
39.7
77.6
58.5
Appendix
Table 2: Average ASR (in %) across source-target model pairs, computed from the off-diagonal entries of the heatmaps of Fig. 5 and Fig. 6 . Bold letters indicate better cross-model transferability .
Fig. 5 : ASR (in % ) with ε=8/255 and εDGF=η8/255 , on ImageNet , for different pretrained (top) and RobustBench (bottom) models. Left to right: proposed attack, AutoAttack, F-PGD . Color intensity indicates higher transferability . We observe that our proposed attack is consistently more transferable than the baselines, as we double the attack level (cf. Fig. 3(a) ), across a variety of models. These results conform with our motivation deriving attacks from a principled methodology, based on structured transforms.
Fig. 6 : ASR ( % ) on CIFAR100 , with ε=16/255 for the baselines and εDGF=ηε for the proposed attack, on different pretrained (top) and RobustBench (bottom) models. Color intensity indicates higher transferability . We notice that our attack (left) is highly transferable, even more than on ImageNet (cf. Fig. 5 ), outperforming the baselines (middle and right) , for both pretrained and robust models.
Fig. 7 : Exemplary spatial (top) and frequency (bottom) representation of the proposed attack and the two baselines , corresponding to the adversarial images of Figure 4 . Similarly to Fig. 1(b) and Fig. 2(b) , we observe that the SF ability of our attack, for both datasets, provides a balanced representation in the transformed space, while the baselines rely mostly on spatial and frequency coordinates, thus missing the flexibility offered by the DGF.
Transfer-based adversarial attacks often transfer poorly across heterogeneous architectures because CNNs favor local textures while Vision Transformers (ViTs) rely on global shapes. We propose Season, a spectrum-aware orthogonal gradient refinement framework for L-infinity transfer attacks against black-box target models on ImageNet, using a white-box surrogate. Season decomposes each update into a low-frequency branch capturing structural cues and a high-frequency branch capturing textures. A low-saliency guidance scheme reallocates high-frequency energy to background regions, preserving foreground structures that ViTs depend on. An orthogonal projection then forces the textural update to lie in the orthogonal complement of the structural direction, mitigating feature interference. As a training-free plug-and-play wrapper, Season enhances eight gradient-stabilization and input-enhancement attacks without modifying their cores. Across eight CNN, ViT, and MLP targets, Season improves transfer success rate by 6.6 percentage points on average and up to 16.0 points over strong baselines under a unified protocol.
Tianyi Wang, Zhenghao Gao, Shengjie Xu
Tongji University Shanghai, China · Huazhong University of Science and Technology · Wuhan LightRead Intelligent Technology Co., Ltd. Wuhan, China
Multi-threat robustness remains a fundamental challenge in deep learning. Although joint adversarial training (JAT) is widely adopted, it suffers from negative transfer under heterogeneous threats, particularly between ℓp-bounded and semantic attacks. Through first-order gradient analysis, we formalize this as gradient incompatibility and theoretically establish the necessity of decoupled optimization. We further reveal that these conflicting threats exhibit separable spectral characteristics in the frequency domain. Motivated by this observation, we propose Threat-aware Frequency Decoupling (TaFD), a two-stage defense framework that reformulates JAT as a frequency-domain divide-and-conquer paradigm. TaFD first discovers latent threat domains via unsupervised clustering of attack spectral prototypes and trains a lightweight classifier for inference-time threat domain identification. Conditioned on the prediction, TaFD employs a Frequency-Conditional Convolution that learns threat-domain-specific spectral masks and routes each sample to the corresponding expert, enforcing structural parameter separation and alleviating optimization conflicts. We validate TaFD on three representative image-classification benchmarks (CIFAR-10, CIFAR-100, and Tiny-ImageNet) and on two representative architectures (the convolutional ResNet and the hybrid-transformer MobileViT). Extensive results demonstrate that TaFD achieves more balanced robustness against heterogeneous attacks than existing JAT and frequency-domain baselines, improving average robust accuracy by approximately 11% over the strongest baseline while maintaining leading clean accuracy.
Mengda Xie, Yiling He, Meie Fang
School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou, Guangdong, China · Information Security Research Group, University College London, London, United Kingdom
Transformation-based adversarial attacks (TAAs) demonstrate strong transferability when deceiving classification models. However, existing TAAs often perform unsatisfactorily or even fail when applied to structured tasks such as semantic segmentation and object detection. Encouragingly, recent studies that categorize transformations into non-spatial and spatial transformations inspire us to address this challenge. We find that for non-structured tasks, labels are spatially non-structured, and thus TAAs are not required to adjust labels when applying spatial transformations. In contrast, for structured tasks, labels are spatially structured, and failing to transform labels synchronously with inputs can cause spatial misalignment and yield erroneous gradients. To address these issues, we propose a novel unified Spatial Alignment Framework (SAF) for highly transferable TAAs on spatially structured tasks, where the TAAs spatially transform labels synchronously with the input using the proposed Spatial Alignment (SA) algorithm. Extensive experiments demonstrate the crucial role of our SAF for TAAs on structured tasks. Specifically, in non-targeted attacks, our SAF degrades the average mIoU on Cityscapes from 24.50 to 11.34, and on Kvasir-SEG from 49.91 to 31.80, while reducing the average mAP of COCO from 17.89 to 5.25.