Defending against Patch-Based and Texture-Based Adversarial Attacks with Spectral Decomposition

Wei Zhang; Xinyu Chang; Xiao Li; Yiming Zhu; Xiaolin Hu

cs.CVApr 12, 2026

Defending against Patch-Based and Texture-Based Adversarial Attacks with Spectral Decomposition

Authors: Wei Zhang, Xinyu Chang, Xiao Li, Yiming Zhu, Xiaolin Hu

Organizations: Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China. · School of Software, Tsinghua University, Beijing 100084, China. · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China. · Department of Computer Science and Technology, Institute for AI, BNRist, IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China ·

Abstract

Adversarial examples present significant challenges to the security of Deep Neural Network (DNN) applications. Specifically, there are patch-based and texture-based attacks that are usually used to craft physical-world adversarial examples, posing real threats to security-critical applications such as person detection in surveillance and autonomous systems, because those attacks are physically realizable. Existing defense mechanisms face challenges in the adaptive attack setting, i.e., the attacks are specifically designed against them. In this paper, we propose Adversarial Spectrum Defense (ASD), a defense mechanism that leverages spectral decomposition via Discrete Wavelet Transform (DWT) to analyze adversarial patterns across multiple frequency scales. The multi-resolution and localization capability of DWT enables ASD to capture both high-frequency (fine-grained) and low-frequency (spatially pervasive) perturbations. By integrating this spectral analysis with the off-the-shelf Adversarial Training (AT) model, ASD provides a comprehensive defense strategy against both patch-based and texture-based adversarial attacks. Extensive experiments demonstrate that ASD+AT achieved state-of-the-art (SOTA) performance against various attacks, outperforming the APs of previous defense methods by 21.73%, in the face of strong adaptive adversaries specifically designed against ASD. Code available at https://github.com/weiz0823/adv-spectral-defense .

Figures & tables

Fig. 1: Spectral analysis of different physically realizable adversarial attacks with Discrete Wavelet Transform (DWT). $D_{1}$ to $D_{6}$ represent the detail (high-frequency) sub-bands obtained from multi-level DWT, with $D_{1}$ being the highest frequency. The amplitudes correspond to absolute values of the wavelet coefficient components at each decomposition level. They capture image edges, textures, and local spatial variations at different scales. The figure highlights abnormal spectral amplitudes caused by physical adversarial perturbations. Error bars: $95\text{\,}\mathrm{\%}$ confidence intervals. Upper left: An adversarial patch (AdvPatch), an adversarial texture (AdvTexture, cropped), and a benign image patch, visualized. Bottom: A sketch of DWT.

Fig. 2: Illustration of the DWT decomposition. Left: The 1D hierarchical decomposition. Right: The outputs of the decomposition on a 2D image, called the 2D DWT pyramid.

Fig. 3: Overview of the pipeline of the proposed ASD method. ASD has a masking module, called ASD Masking , to detect and mask out possible adversarial regions with $n$ -level DWT decomposition. Then the preprocessed images are fed into the AT detector to offer robustness against low-amplitude adversarial patterns. By setting DWT level to $n_{1},n_{2},n_{3},\ldots$ , ASD level aggregation is finally used to adapt to different object sizes in the images. The block on the right shows the details of ASD Masking, which is based on spectral decomposition.

Panel of Figure 4 — Fig. 4: Visualization of adversarial examples with patch-based or texture-based attacks, and the images processed by ASD. (a) AdvPatch [ 5 ] example. (b) AdvPatch processed by ASD. (c) AdvTexture [ 11 ] example. (d) AdvTexture processed by ASD.

Fig. 5: The DWT amplitude of different physically realizable adversarial examples against the adversarially trained Faster R-CNN.

D_{1}

D_{6}

correspond to different frequency levels, from higher frequency to lower frequency. Error bar:

95\text{\,}\mathrm{\%}

confidence interval.

Fig. 6: Frequency-amplitude division of ASD and AT.

Defense	Adaptive type	AdvPatch
ASD	Direct	71.68
	Loss	71.85
	Range	58.78
ASD + AT	Direct	90.04
	Loss	88.00
	Range	92.71

TABLE I: AP 50 (%) of Faster R-CNN against different types of adaptive attacks designed for ASD.

		Patch		Texture		Imperceptible
Defense	Clean	AdvPatch [ 5 ]	AdvTshirt [ 8 ]	AdvTexture [ 11 ]	AdvCaT [ 6 ]	NumbOD [ 22 ]	LGP [ 23 ]	Average
$l_{\infty}$ -AT [ 30 ]	95.96	55.46	9.59	29.08	39.59	93.31	94.38	53.57
FD-JPEG [ 31 ] + AT	89.92	52.59	8.04	25.98	39.17	84.44	84.22	49.07
IDBD [ 47 ] + AT	95.28	48.24	8.48	35.80	32.55	92.25	93.07	51.73
SAC [ 14 ] + AT	95.96	55.46	9.01	26.05	40.98	93.21	94.50	53.20
Jedi [ 16 ] + AT	91.16	72.04	30.17	28.59	40.65	91.62	92.40	59.25

TABLE II: AP 50 (%) of adversarially trained Faster R-CNN [ 42 ] against physically realizable (patch, texture) and imperceptible adversarial attacks in adaptive attack settings.

		Patch		Texture
Defense	Clean	AdvPatch	AdvTshirt	AdvTexture	AdvCaT	Average
$l_{\infty}$ -AT	93.87	30.54	7.17	30.07	18.33	21.53
IDBD + AT	93.65	29.08	6.97	27.27	17.82	20.28
FD-JPEG + AT	88.64	28.62	6.16	26.55	16.28	19.40
SAC + AT	93.87	30.97	7.36	26.89	19.29	21.13
Jedi + AT	90.35	37.20	18.16	21.52	9.48	21.59

TABLE III: AP 50 (%) of adversarially trained FCOS [ 44 ] against physically realizable adversarial attacks in the adaptive attack settings. The same conventions are used as in Table II .

		Patch		Texture
Defense	Clean	AdvPatch	AdvTshirt	AdvTexture	AdvCaT	Average
$l_{\infty}$ -AT	66.25	38.32	43.50	57.39	51.42	47.66
IDBD + AT	66.18	36.22	20.51	58.45	53.01	42.05
FD-JPEG + AT	58.73	34.13	19.97	52.06	46.13	38.07
SAC + AT	$\mathbf{66.26}$	38.16	22.20	52.89	52.68	41.48
Jedi + AT	49.27	43.67	25.89	55.06	55.38	45.00

TABLE IV: AP 50 (%) of adversarially trained YOLOv8 [ 45 ] against patch-based and texture-based attacks in the adaptive attack settings. The same conventions are used as in Table II .

Defense	AdvTexture	AdvCaT
None	0.23	5.94
IDBD	6.97	7.92
FD-JPEG	0.19	18.81
SAC	60.80	5.94
Jedi	80.36	92.85
APE	0.73	9.80

TABLE V: AP 50 (%) of Faster R-CNN [ 42 ] against texture-based attacks in the physical world.

Patch	V2	V3	V4	V4t	FR
AP 50 (%)	92.67	92.07	89.62	90.33	90.56

TABLE VI: AP 50 (%) of Faster R-CNN with ASD+AT against different NatPatch patches. Patches are denoted with the names in the original paper [ 10 ] .

Configuration of ASD	AP 50 (%)
Full configuration	$\mathbf{88.00}$
Without AT	58.78 ( $\downarrow$ 29.22)
Without level aggregation ( $k=1$ )	78.91 ( $\downarrow$ 9.09)
Single-spectral masking ( $n=1,k=1$ )	75.67 ( $\downarrow$ 12.33)

TABLE VII: Ablation study on ASD, evaluated against AdvPatch attack. Target model is Faster R-CNN.

$\theta$	0.05	0.10	0.15	0.17	0.20	0.30
AP 50 (%)	71.21	81.99	86.91	88.00	87.71	80.91

TABLE VIII: AP 50 (%) of adversarially trained Faster R-CNN with ASD with different threshold

\theta

, evaluated against AdvPatch attack.

$n$	1	2	3	4	5
Without level aggregation	75.67	81.46	78.91	67.30	58.99
With level aggregation	74.12	83.64	88.00	88.52	88.34

TABLE IX: AP 50 (%) of adversarially trained Faster R-CNN with ASD with different expansion levels

n

, evaluated against AdvPatch attack.

Transform	DWT	FFT	DCT
AP 50 (%)	88.00	55.50	55.39

TABLE X: AP 50 (%) of adversarially trained Faster R-CNN with ASD and different time-frequency transformation methods, evaluated against AdvPatch attack.

Defense	AdvPatch	AdvTexture	NumbOD
None	36.99	0.21	12.25
ASD	58.78	15.86	18.49
AT	55.46	29.08	93.31
ASD+AT	88.00	70.08	93.23

TABLE XI: AP 50 (%) of differently trained Faster R-CNN with ASD against attacks.

		Patch		Texture
Defense	Clean	AdvPatch	AdvTshirt	AdvTexture	AdvCaT	Average
None	96.21	36.99	8.93	0.21	0.32	11.61
IDBD	95.09	42.99	8.28	2.13	0.34	13.44
FD-JPEG	90.34	30.84	6.66	0.15	0.38	9.51
SAC	96.21	57.72	35.53	0.25	0.67	23.54
Jedi	92.67	$\mathbf{64.40}$	27.36	2.30	0.70	23.69

TABLE XII: AP 50 (%) of standardly trained Faster R-CNN against patch-based and texture-based attacks in the adaptive attack settings. The same conventions are used as in Table II .

Defense	Tench	Balloon	Umbrella	Throne	Teddy bear
AT	56.22	32.84	31.54	75.80	16.30
AT+ASD	99.42	78.74	87.50	99.86	99.96

TABLE XIII: Accuracy (%) of ResNet-50 [ 48 ] with defenses against Adversarial Patch [ 3 ] targeting different ImageNet [ 49 ] classes.

Defense	Latency ( $\mathrm{m}\mathrm{s}\mathrm{/}\mathrm{i}\mathrm{m}$ )	AP 50 (%)
None	22.5	55.46
FD-JPEG	28.2	52.59
ASD (without level agg.)	23.3	78.91
ASD (full)	56.8	88.00

TABLE XIV: Inference latency per image of Faster R-CNN with different frequency-based defense methods applied.

Adversarial Robustness

Adversarial robustness studies how well AI systems withstand deliberately crafted inputs, training-data corruption, and other attacks.

See all 479 papers

Detection-Guided Adaptive Purification with Diffusion Models for Robust Audio Deepfake Detection
Oct 7, 2026Muhammed Salih Kayhan, Qiben YanAudio Deepfake Detection Robustness Audio Deepfake Detection
Michigan State University, East Lansing, MI, USA
Detecting Adversarial Data via Provable Adversarial Noise Amplification
May 4, 2026Furkan Mumcu, Yasin YilmazSpectral Regularization Adversarial Robustness
University of South Florida
Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerability
Mar 26, 2026Jiaming Liang, Chi-Man PunNeural Network Robustness Adversarial Robustness
University of Macau, Macau, China

Oct 7, 2026J/K move · Enter open · S save

Detection-Guided Adaptive Purification with Diffusion Models for Robust Audio Deepfake Detection

Muhammed Salih Kayhan, Qiben Yan

Michigan State University, East Lansing, MI, USA

Audio Deepfake Detection Robustness Audio Deepfake Detection Adversarial Robustness Adversarial Purification Adversarial Example Detection Adversarial Attacks

Open paper AskPDF arXiv

Audio deepfake detectors remain vulnerable to adversarial perturbations that suppress the acoustic cues used for detection, allowing manipulated utterances to evade the detector. Although existing defenses can improve robustness, they require retraining the detector or introduce additional distortion. Diffusion-based purification instead leaves the pretrained detector unchanged, but existing methods use the same purification strength for all inputs, creating a trade-off between removing adversarial perturbations and preserving the subtle spoofing cues needed for detection. In this paper, we propose Detection-Guided Adaptive Purification (DGAP), a diffusion-based defense that adjusts purification strength per input. Building on the observation that a light purification perturbs the detector score of an adversarial input far more than that of a benign one, the framework uses the resulting score shift as a reference-free indicator of adversarial manipulation. Inputs with small shifts are passed unchanged, whereas flagged inputs undergo stronger purification before final detection. We evaluate the framework against three adversarial attack settings across three deepfake detectors, and compare it with nine existing defenses. Our results show that DGAP achieves the best defense performance across all detectors while leaving benign inputs nearly unaffected, and remains effective under the defense-aware adaptive attack.

Abstract

Adversarial Robustness

Explore similar work

Detection-Guided Adaptive Purification with Diffusion Models for Robust Audio Deepfake Detection

Detecting Adversarial Data via Provable Adversarial Noise Amplification

Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerability