Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation

Kaichao Jiang; Changtao Miao; Baiqi Wu; Zhiyuan Lu; Kang Yang; Peiwei Zhao; Junchi Chen; Yunfeng Diao

cs.AIAug 27, 2026

Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation

Authors: Kaichao Jiang, Changtao Miao, Baiqi Wu, Zhiyuan Lu, Kang Yang, Peiwei Zhao, Junchi Chen, Yunfeng Diao, +3 more

Organizations: University of Science and Technology of China · Independent Researcher · Zhejiang University · Hefei University of Technology

Abstract

Recent audio-video generators increasingly support joint conditioning on text, images, audio, and video. These capabilities also enable attacks that exploit cross-modal interactions or obscure harmful intent to bypass safeguards and induce harmful audio-video outputs. However, existing generation-safety benchmarks have not kept pace with these advances, providing limited coverage of multimodal input combinations and obscured attack intents. To address these gaps, we introduce Multi2AV-Safety, the first full-coverage red-team benchmark for multimodal-to-audio-video generation, comprising 11,024 attack instances across all 11 non-singleton T/I/A/V conditioning configurations, 4 attack-intent categories, and 5 harm categories. Our evaluation of recent state-of-the-art models, including four multimodal-conditioned audio-video generators and eight safety guards, reveals substantial vulnerabilities in both generation and safeguarding, with multimodal compositional risk and obscured attack-intent risk emerging as two complementary challenges. Guided by these findings, we introduce PerceptGuard, an omni-modal guard integrating compositional-risk and attack-intent supervision through structured risk perception learning. By jointly training rationale generation and safety classification, it learns shared risk representations that enable a safety head to make efficient predictions at inference without rationale decoding, while retaining the ability to generate explanations on demand. Across 34 safety benchmarks, PerceptGuard combines SOTA multimodal safety detection with highly competitive unimodal performance, strengthening input-side safeguards against multimodal attacks on omni models. In particular, it improves safeguarding against the above risks, achieving an overall recall of 86.06% on Multi2AV-Safety and outperforming GuardReasoner-Omni by 14.56%.

Figures & tables

Input

Attack intent

Harm structure

Benchmark / Dataset

Output

Direct

Jail.

Adv.

Temp.

Full

Diluted

Composed

I2P ( Schramowski et al., 2023 )

\checkmark

–

\checkmark

–

N/A

T2ISafety ( Li et al., 2025 )

\checkmark

–

\checkmark

–

N/A

T2I-RiskyPrompt ( Zhang et al., 2026 )

\checkmark

–

\checkmark

\checkmark

\checkmark

–

N/A

JailbreakDiffBench ( Jin et al., 2025 )

\checkmark

–

I/V

–

\checkmark

\checkmark

–

N/A

MPJ ( Liu et al., 2025 )

\checkmark

–

\checkmark

–

N/A

Table 1: Comparison of representative safety benchmarks for generative models. Full, Diluted, and Composed denote harmful multimodal requests with all, some but not all, or none of their individual modality inputs harmful in isolation, respectively.

Figure 2: Construction pipeline of Multi2AV-Safety . Seeds for four attack-intent categories and their benign counterparts are realized across all 11 conditioning configurations, then validated through model screening and expert review.

Config.	Attack intent counts (#)					Generator ASR (%) $\downarrow$				Input-side Guard Recall (%) $\uparrow$
Direct	Jail.	Adv.	Temp.	$N$		[-1pt]	[-1pt]	[-1pt] davinci	Qwen3-Omni	GR-Omni	Ours

T+I	900	749	100	100	1,849	86.75	37.37	76.85	81.48	58.46	71.07	85.18
T+A	900	400	300	100	1,700	89.24	52.59	–	70.07	66.82	70.00	83.12
T+V	900	200	100	100	1,300	92.00	41.85	76.08	–	76.00	68.85	80.38
I+A	900	400	100	100	1,500	99.07	20.93	64.87	85.36	55.53	78.67	96.53

Table 2: Multimodal-conditioned audio-video generation safety and omni-guard detection across multimodal input combinations in Multi2AV-Safety .

N

denotes the number of requests, and Ours denotes PerceptGuard. Non-omni guard results are in Appendix E.6 .

Figure 3: Guard detection of compositional risks. Across all panels, the two-, three-, and four-modal requests are derived from a shared pool of seed scenarios. (a) Recall on Composed Harm ( $k=0$ ) across input modality counts. (b–d) Recall on Direct attacks with two, three, and four modalities, respectively, grouped by the number of individually harmful modality inputs $k$ . In (b–d), $0<k<|S|$ denotes Diluted Harm , while $k=|S|$ provides the Full Harm reference.

Figure 4: Overview of PerceptGuard , which learns shared safety representations through compositional risk and attack intent supervision, enabling direct prediction and on-demand explanations.

Training objectives				Supervised outputs		32-task F1 (%) $\uparrow$		Omni recall (%) $\uparrow$
$\mathcal{L}_{\mathrm{cls}}$	$\mathcal{L}_{\mathrm{rel}}$	$\mathcal{L}_{\mathrm{gen}}$	$\mathcal{L}_{\mathrm{intent}}$	Rationale generation	Intent recognition	Request	Response	Omni- SafetyBench	Multi2AV- Safety
$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	87.44	74.05	99.35	71.43
$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$	$\times$	88.60	73.39	99.13	76.24
$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\times$	89.01	72.56	99.37	82.18
[0pt][0.2pt] $\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	88.98	74.83	99.12	86.06

Table 5: Ablation of PerceptGuard. F1 is sample-weighted by task; recall uses full-benchmark denominators. All variants use safety-head inference without rationale decoding.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Axis	Types and variations
Modality	T: existing safety datasets, adapted and refined with Claude Opus 4.6; I: Stable Diffusion 1.5 ( Rombach et al., 2022 ) / HiDream-O1-Image ( Cai et al., 2026 ) ; A: TTS by CosyVoice2 ( Du et al., 2024 ) / TTS-1 ( OpenAI, 2023 ) ; V: LTX-2 ( HaCohen et al., 2026 ) / daVinci-MagiHuman ( SII-GAIR et al., 2026 ) .
Conditioning	2-modal: T+I, T+A, T+V, I+A, I+V, A+V; 3-modal: T+I+A, T+I+V, T+A+V, I+A+V; 4-modal: T+I+A+V.
Attack intent	Direct: explicit harmful intent; Obscured: Jailbreak, Adversarial, and Temporal intents.
Harm evidence	Composed Harm ( $k=0$ ): individually benign inputs convey harm in the complete multimodal request; Diluted Harm ( $1\leq k<\|S\|$ ): explicit harmful evidence is embedded in benign context; Full Harm ( $k=\|S\|$ ): all active inputs are harmful in isolation.

Appendix

Table B.1: Construction taxonomy of Multi2AV-Safety , covering modalities, conditioning, attack intents, and harm evidence.

Figure 5: A successful attack scenario across four audio-video generators. Harmful text is combined with benign media, forming Diluted Harm . The shared attack scenario elicits harmful outputs from all four generators under their respective conditioning configurations. Representative output frames are shown with sensitive regions masked.

Table 10

Table 11

Explore similar work

The Price of Consistency: Exploiting Visual Anchors for Multimodal Jailbreaking in Video Generation
Sep 7, 2026Peng Li, Qianqian Xu, Yangbangyan Jiang +2Image-to-Video Generation Multimodal Jailbreak Attacks
School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, China +1
VPA-Guard: Defending and Benchmarking Image-to-Video Generation Against Visual Prompt Attacks
Jun 24, 2026Yining Sun, Haoyu Kang, Jiajun Wu +7Image-to-Video Generation Video Generation
1Tsinghua University · 3Central South University · 4South China Normal University +3
SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation
May 31, 2026Yingzi Ma, Xiaogeng Liu, Yawen Zheng +1Image-to-Video Generation Text-to-Video Generation
University of Wisconsin-Madison, Madison · Tsinghua University · Johns Hopkins University

Sep 7, 2026, cs.CVJ/K move · Enter open · S save

The Price of Consistency: Exploiting Visual Anchors for Multimodal Jailbreaking in Video Generation

Peng Li, Qianqian Xu, Yangbangyan Jiang, Zhipeng Yu+1

School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, China+1

Image-to-Video Generation Multimodal Jailbreak Attacks Controllable Video Generation

Open paper AskPDF arXiv

The rapid evolution of video generation has shifted the paradigm from pure text-driven to multi-conditional controllable generation, with reference images now widely adopted as conditional inputs to achieve superior spatiotemporal consistency. While these reference images serve as powerful visual anchors that significantly enhance controllability, their impact on safety remains largely unexplored. In this work, we reveal the visual anchoring effect: by enforcing consistency, the mechanism prevents the generated content from drifting away from the original harmful intent, thereby eliminating the model's natural safety escape route from harmful to benign content. Consequently, visual anchors inherently increase the safety risk---this is the price of consistency. Building on this insight, we propose Decoupling Intent via Visual Anchors (DIVA), a training-free multimodal jailbreak framework for video generation that exploits this vulnerability. DIVA decouples harmful intent into a static visual anchor image and a dynamic motion text prompt, and employs dual-criteria selection to balance attack stealthiness with semantic preservation. Extensive experiments across various leading commercial platforms and mainstream open-source video generation models demonstrate that DIVA achieves a substantially higher Attack Success Rate than existing text-only methods. To facilitate future research, we additionally contribute TI2VSafetyBench, the first safety benchmark for multi-conditional video generation.

Abstract

Explore similar work

The Price of Consistency: Exploiting Visual Anchors for Multimodal Jailbreaking in Video Generation

VPA-Guard: Defending and Benchmarking Image-to-Video Generation Against Visual Prompt Attacks

SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation