A Survey on Adversarial Attacks and Defenses for Diffusion Models Across Multiple Modalities

Ozgur Kara; Tarik Can Ozden; Furkan Horoz; Zeqian Long; Haotian Xue; Yipu Chen; Oguzhan Akcin; Yongxin Chen

cs.CVAug 28, 2026

A Survey on Adversarial Attacks and Defenses for Diffusion Models Across Multiple Modalities

Authors: Ozgur Kara, Tarik Can Ozden, Furkan Horoz, Zeqian Long, Haotian Xue, Yipu Chen, Oguzhan Akcin, Yongxin Chen,

Organizations: University of Illinois Urbana-Champaign · Stanford University · Georgia Institute of Technology · The University of Texas at Austin

Abstract

Diffusion models have become the dominant family of generative models in the visual domain. However, their widespread public availability enables misuse at scale, motivating a rapidly growing body of research on adversarial attacks and defenses. This survey provides, to our knowledge, the first unified review of this literature across three visual modalities: image, video, and 3D. We introduce a comprehensive, task-centric taxonomy: we first divide the literature by modality; within each modality, we separate methods into attacks and defenses, and then group them by the generative task they target, presenting them chronologically within each task. Moreover, we provide an in-depth analysis of their evaluation settings, consolidating the datasets, metrics, and benchmarks used to assess them. We conclude by identifying several open challenges and outlining concrete future research directions. Project Webpage: https://github.com/ozgurkara99/awesome-adv-attack-defense-on-diffusion

Figures & tables

Figure 2: Diffusion-model immunization across modalities. Each row represents an Alice–Eve–Bob scenario: Alice publishes content $x$ (col. 1); Bob applies an imperceptible cloak $\delta$ under budget $\|\delta\|_{\mathcal{X}}\!\leq\!\epsilon$ (col. 2); Eve executes her generative pipeline $D_{\theta}(z,\,x\oplus\delta,\,c)$ on the protected asset (col. 3), yielding a degraded or corrupted output (col. 4).

Modality	Eve’s Pipeline (Target Task)	Bob’s Cloak Space ( $\delta$ )	Budget ( $\epsilon$ )	Entry Point ( $\phi$ )
Image	• Personalization • Style / copyright • Editing / inpainting • Face-swap / erasure	• Additive pixel noise ( $\ell_{\infty}$ )	$\{4,8,16\}/255$	• VAE encoder $\mathcal{E}$ (Latent) • Identity map (Pixel) • Patch token (DiT)
Video	• I2V animation • Talking-head • V2V editing • T2V safety	• Additive frame noise ( $\ell_{\infty}$ ) • Optical flow shift ( $\ell_{2}$ )	$\{8,16,32\}/255$	• Temporal-attn. encoder • Spatio-temporal tokens
3D	• Content editing • Poisoning / comp. cost • Reconstruction	• 3D-representation params. offsets ( $\Delta$ ) • Rendered-view cloak (lifted)	Method-specific	• 3D representation params. • Differentiable renderer $\mathcal{R}$

Table 1: The adversarial framework of ( 3 ), instantiated per modality. Each row represents one Alice–Eve–Bob scenario: Bob’s cloak

\delta

, bounded by budget

\epsilon

, protects Alice’s content against Eve’s pipeline, entering through the structural feature map

\phi

Figure 3: Task-centric taxonomy of adversarial attacks and defenses for diffusion models across (a) image, (b) video, and (c) 3D. Methods are grouped by generative task within each modality; image and 3D defenses are collected in separate branches, while video counter-defenses remain nascent and are discussed in the text (§ 3.2 ). Red marks attacks and green marks defenses.

Method	Venue	Evaluation Sources	Target Models	IA	FA	TR	RB
I2V
UVCG [ 71 ]	arXiv’24	DAVIS subset	SVD, TokenFlow, Text2Video-Zero	✓	✗	✓	✗
I2VGuard [ 35 ]	CVPR’25	Manually curated	SVD, CogVideoX, ControlNeXt	✓	✗	✗	✗
DORMANT [ 205 ]	USENIX’25	TikTok, Champ, UBC Fashion, TED Talks	Animate Anyone, MagicAnimate, MagicPose, MusePose, Champ, MuseV, UniAnimate, ControlNeXt	✓	✓	✓	✓
Vid-Freeze [ 23 ]	arXiv’25	Manually curated	SVD, CogVideoX	✓	✗	✗	✓
Anti-I2V [ 150 ]	CVPR’26	CelebV-Text, UCF101	CogVideoX, OpenSora, DynamiCrafter	✓	✗	✓	✓

Table 2: Video attack and defense methods.

Method	Venue	Evaluation Sources	Target Models	IA	FA	TR	RB
Editing
Image2Multiview [ 141 ]	ICME’25	Google Scanned Objects, Internet images	Zero123++, Wonder3D	✓	✗	✓	✓
DEGauss [ 97 ]	NeurIPS’25	NeRF-Art, Mip-NeRF 360	GaussianEditor, DGE, DreamCatalyst, EditSplat	✓	✓	✗	✗
AdLift [ 46 ]	ICML’26	IN2N, NeRF-Art, BlendedMVS scenes	IP2P, Instruct-GS2GS, DGE	✓	✓	✓	✓
Poisoning / cost
Poison-splat [ 92 ]	ICLR’25	NeRF-Synthetic, Mip-NeRF 360, Tanks & Temples	Orig. 3DGS, Scaffold-GS	✓	✓	✓	✓

Table 3: 3D attack and defense methods.

Dataset	Content	Scale
Image
Faces & identity: source portraits for personalization and face-swap cloaks
CelebA [ 89 ]	in-the-wild celebrity faces; 40 binary attributes, 5 landmarks	202,599 imgs, 10,177 IDs
VGGFace2 [ 12 ]	celebrity/public-figure faces; large pose and age variation	3.31M imgs, 9,131 IDs
FFHQ [ 54 ]	high-quality aligned Flickr faces; wide demographic coverage	70,000 imgs at $1024^{2}$
LFW [ 49 ]	unconstrained faces for pair-matching verification	13,233 imgs, 5,749 people

Table 4: Datasets and benchmarks used across the surveyed literature , grouped by modality and by their role in attack/defense evaluation. Detailed descriptions follow in the text.

Metric	Signature (inputs $\to$ output)	What it measures	Goal
Output side: effectiveness (Eve’s generation should degrade)
FID	$(\{y\},\,\{y^{\mathrm{ref}}\})\to\mathbb{R}_{\geq 0}$	Fréchet distance between Gaussians fitted to Inception features of the generated and reference sets; distribution-level realism	$\uparrow$
LPIPS	$(y,\,y^{\mathrm{clean}})\to\mathbb{R}_{\geq 0}$	weighted distance between deep-network activations of an image pair; perceptual dissimilarity	$\uparrow$
SSIM / PSNR	$(y,\,y^{\mathrm{clean}})\to[0,1]$ / dB	structural similarity (luminance, contrast, structure) / log-scaled peak-signal-to-MSE ratio; low-level fidelity	$\downarrow$
CLIP-I / CLIP-T	$(y,\,x)$ or $(y,\,c)\to[-1,1]$	cosine similarity of CLIP embeddings; semantic faithfulness of the output to the source subject (-I) or to the prompt (-T)	$\downarrow$
FDFR	$\{y\}\to[0,1]$	fraction of generated images in which a face detector finds no face	$\uparrow$

Table 5: Definitions of the evaluation metrics. Each metric is written as a function of its inputs.

x

is Alice’s clean asset and

x\oplus\delta

its protected version;

c

is the conditioning (prompt, subject, or audio

a

);

y=D_{\theta}(z,\,x\oplus\delta,\,c)

is Eve’s output on the protected asset and

y^{\mathrm{clean}}=D_{\theta}(z,\,x,\,c)

her output on the clean asset;

V

is a generated video and

\{\mathcal{R}_{v}\}_{v}

are renders of a 3D asset from viewpoints

v

. The Goal column gives the direction indicating successful protection : degraded generation (top block), an invisible cloak (middle block), and cheap protection (bottom block).

Adversarial Attacks on Diffusion Models

See all 15 papers

Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models
Jun 25, 2026Abrar Alotaibi, Moataz AhmedAdversarial Attacks on VLMs Adversarial Attacks
Information and Computer Science Department, King Fahd University of Petroleum & Minerals, Dhahran, 31261, Saudi Arabia · College of Computer Science and Information Technology, Imam Abdulrahman Bin Faisal University, Dammam, 31441, Saudi Arabia · SDAIA-KFUPM Joint Research Center for Artificial Intelligence, King Fahd University of Petroleum & Minerals, Dhahran, 31261, Saudi Arabia
Fragile Reconstruction: Adversarial Vulnerability of Reconstruction-Based Detectors for Diffusion-Generated Images
Apr 14, 2026Haoyang Jiang, Mingyang Yi, Shaolei Zhang +4Adversarial Attacks on AI-Generated Image Detectors AI-Generated Image Detection
Renmin University of China, Beijing, China · Tencent Inc., Shenzhen, China
Anti-I2V: Safeguarding your photos from malicious image-to-video generation
Mar 25, 2026Duc Vu, Anh Nguyen, Chi Tran +1Image-to-Video Generation Video Diffusion Models
Qualcomm AI Research†

Jun 25, 2026J/K move · Enter open · S save

Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models

Abrar Alotaibi, Moataz Ahmed

Information and Computer Science Department, King Fahd University of Petroleum & Minerals, Dhahran, 31261, Saudi Arabia · College of Computer Science and Information Technology, Imam Abdulrahman Bin Faisal University, Dammam, 31441, Saudi Arabia · SDAIA-KFUPM Joint Research Center for Artificial Intelligence, King Fahd University of Petroleum & Minerals, Dhahran, 31261, Saudi Arabia

Adversarial Attacks on VLMs Adversarial Attacks Adversarial Attacks on LLMs Adversarial Robustness Adversarial Robustness of Language Models

Open paper AskPDF arXiv

Adversarial evaluation of AI systems has matured along four largely disconnected tracks: diffusion-based attacks on text and large language models (LLMs), diffusion-based attacks on image classifiers, jailbreak pipelines against vision-language models, and diffusion-based input purification defenses. Each has developed its own vocabulary, threat models, and benchmarks, with denoising diffusion models emerging as a shared generative mechanism whose recipes are now actively ported between communities. This survey performs an information-fusion exercise at the meta-research level: we integrate these four tracks into a single conceptual framework with a unified taxonomy, evaluation criteria, and research agenda, focusing on the LLM-side slice. We catalog fifty published papers across four scope areas (text/LLM, image classifier, vision-language model, defense), plus four diffusion-LLM-as-victim entries and ten non-diffusion baselines against which any new attack must be compared. We propose a six-class taxonomy of diffusion roles in adversarial pipelines, augmented by a threat-model axis recording attacker knowledge, query budget, and target accessibility, and apply a five-dimension framework (attack success rate, transferability, query budget, perplexity, defense-evasion) uniformly across modalities. The review adopts a dual attacker-defender perspective: alongside the attack catalog we cover four diffusion-based defenses that form the natural evaluation backdrop for new attacks. Our critical analysis identifies five recurring weaknesses of the current LLM-side literature, and we close with a research agenda of open questions and concrete experimental designs. The companion catalog and spreadsheet are released with the paper. We are explicit that this is a narrative review with quality assessment, not a PRISMA-compliant systematic review, and discuss the implications for replication.

Abstract

Adversarial Attacks on Diffusion Models

Explore similar work

Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models

Fragile Reconstruction: Adversarial Vulnerability of Reconstruction-Based Detectors for Diffusion-Generated Images

Anti-I2V: Safeguarding your photos from malicious image-to-video generation