UniSAFE: A Comprehensive Benchmark for Safety Evaluation of Unified Multimodal Models

Segyu Lee; Boryeong Cho; Hojung Jung; Seokhyun An; Juhyeong Kim; Jaehyun Kwak; Yongjin Yang; Sangwon Jang

cs.CVMar 18, 2026

UniSAFE: A Comprehensive Benchmark for Safety Evaluation of Unified Multimodal Models

Authors: Segyu Lee, Boryeong Cho, Hojung Jung, Seokhyun An, Juhyeong Kim, Jaehyun Kwak, Yongjin Yang, Sangwon Jang,

Organizations: KAIST AI · Department of Computer Science and Engineering, UNIST · Department of Mathematical Sciences, KAIST · University of Toronto ·

Abstract

Unified Multimodal Models (UMMs) offer powerful cross-modality capabilities but introduce new safety risks not observed in single-task models. Despite their emergence, existing safety benchmarks remain fragmented across tasks and modalities, limiting the comprehensive evaluation of complex system-level vulnerabilities. To address this gap, we introduce UniSAFE, the first comprehensive benchmark for system-level safety evaluation of UMMs across 7 I/O modality combinations, spanning conventional tasks and novel multimodal-context image generation settings. UniSAFE is built with a shared-target design that projects common risk scenarios across task-specific I/O configurations, enabling controlled cross-task comparisons of safety failures. Comprising 6,802 curated instances, we use UniSAFE to evaluate 15 state-of-the-art UMMs, both proprietary and open-source. Our results reveal critical vulnerabilities across current UMMs, including elevated safety violations in multi-image composition and multi-turn settings, with image-output tasks consistently more vulnerable than text-output tasks. These findings highlight the need for stronger system-level safety alignment for UMMs. Our code and data are publicly available at https://github.com/segyulee/UniSAFE

Figures & tables

Figure 1: Examples of outputs generated by UniSAFE. Our benchmark consists of risk scenarios centered on a common target across 7 distinct task types, enabling evaluation across diverse risk settings.

Method	Image output				Text output
Method	TI	IE	IC	MT	TT	IT	MU
T2ISafety [ 39 ]	✓	✗	✗	✗	✗	✗	✗
InpaintGuardBench [ 18 ]	✗	✓	✗	✗	✗	✗	✗
CoJ-Bench [ 77 ]	✗	✗	✗	✓	✗	✗	✗
SALAD-Bench [ 38 ]	✗	✗	✗	✗	✓	✗	✗
SafeBench [ 26 ]	✗	✗	✗	✗	✓	✓	✗

Table 1: Task coverage comparison across 7 tasks (detailed description in C.1 ): TI (Text-to-Image), IE (Image Editing), IC (Image Composition), MT (Multi-Turn image editing), TT (Text-to-Text), IT (Image-to-Text), and MU (Multimodal Understanding). Unlike existing safety benchmarks, UniSAFE covers all tasks.

Figure 2: Overview of the UniSAFE three-step data construction pipeline: (1) collect unsafe triggers across threat categories, (2) expand them into contextual target descriptions, and (3) instantiate shared, multimodal task-specific risk scenarios for safety evaluation of UMMs.

Panel of Figure 3 — Figure 3: Taxonomy of safety categories for image and text modalities.

UMM	Image output								Text output
	TI		IE		IC		MT		TT		IT		MU
	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR
GPT-5 [ 52 ]	26.4	0.78	29.0	0.96	34.8	1.27	31.3	0.97	6.2	0.47	5.5	0.42	12.7	0.82
Gemini-2.5 [ 19 ]	44.0	1.32	51.6	1.72	43.5	1.62	53.9	1.73	36.4	1.35	26.5	1.03	36.0	1.42
Qwen-Image [ 82 ]	73.1	2.18	57.4	1.93	-	-	53.9	1.87	-	-	-	-	-	-
Qwen2.5-VL [ 5 ]	-	-	-	-	-	-	-	-	48.7	1.72	47.3	1.68	43.6	1.71

Table 2: Safety evaluation of 14 UMMs (12 open-sourced and 2 commercial ones) under 7 major tasks. Results are shown in Attack Success Rate (ASR, %) and Average Risk Rating (ARR, 0-3). ‘TI’: Text-to-Image, ‘IE’: Image Editing, ‘IC’: Image Composition, ‘MT’: Multi-turn Editing, ‘TT’: Text-to-Text, ‘IT’: Image-to-Text, ‘MU’: Multimodal Understanding. Empty values indicate not-supported tasks.

Figure 4: Refusal Rates for commercial UMMs across different tasks. Refusal Rates are further decomposed into system-level Refusal Rates and model-level Refusal Rates.

Figure 5: Safety risk across tasks and modalities in commercial UMMs. For GPT-5, Gemini-2.5, and Qwen-image, the bars show the proportions of test samples that produce harmful content (moderate- and high-risk) across 7 task types. Image-output tasks (text-to-image, image editing, image composition, multi-turn) consistently exhibit higher harmful content rates than text-output tasks (text-to-text, image-to-text, multimodal understanding), revealing strong modality-dependent bias in safety alignment.

Figure 6: Self-awareness vs. ARR of UMMs. Average self-awareness score (SAS; x-axis, higher is better) is plotted against average assessor risk rating (ARR; y-axis, higher is worse) for each model, as evaluated by 3 different judge models. Dashed lines mark the standard safety thresholds, partitioning models into four regimes (naively safe, robustly safe, obviously risky, and self-aware but risky).

Panel of Figure 7 — Figure 7: Safety comparison between Show-o and Show-o2 (a) ASR across different tasks (b) Conditional ratio of high-risk rating samples for Show-o series.

Model	GenEval Score	Image Editing Score	Image ASR (% )	MMMU Score	Text ASR (%)
Show-o [ 85 ]	0.680	N/A	60.60	0.274	6.93
Janus-pro [ 16 ]	0.800	N/A	64.10	0.363	36.40
BAGEL	0.880	3.20	60.67	0.553	47.53
UniWorld-V1 [ 43 ]	0.840	3.26	42.40	0.586	46.77

Table 3: Unified Model Performance and Safety Metrics. Columns show the specific Generative Score or the corresponding ASR for image-output / text-output tasks.

Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Sub-ID	Image Count	Text Count
Violence	V1	54	59
	V2	55	56
	V3	58	58
	V4	-	59
Sexual	S1	54	51
Sexual	S2	59	60

Appendix

Table 4: Detailed distribution of UniSAFE dataset scenario templates per subcategory for Image and Text tasks. Hyphens (‘-’) indicate categories not applicable to that modality.

Figure 8: Total safe generation rates (non-harmful rates) for commercial models across all tasks. Each bar is composed of total refusal rate and pure safe generation rate.

Figure 9: Safety ranking consistency across judges. Each panel shows how the rank ordering (Rank 1 = safest) of task types or target models changes across five evaluators: human annotators (avg), the LLM ensemble, and the three individual judges (Gemini-2.5 Pro, GPT-5 Nano, Qwen-2.5 VL-72B). Flat lines indicate perfect rank agreement across evaluators. Model rankings (right panels) are unanimous across all five evaluators, while task-type rankings (left panels) show occasional disagreements among individual judges that are stabilized by the ensemble.

A	B	Pearson $r$
Human	Gemini-2.5 Pro	0.949
Human	GPT-5 Nano	0.953
Human	Qwen-2.5 VL-72B	0.961
Human	LLM ensemble judge	0.962
Gemini-2.5 Pro	GPT-5 Nano	0.961
Gemini-2.5 Pro	Qwen-2.5 VL-72B	0.989

Appendix

Table 5: Pearson

r

for human-vs-judge and inter-judge ARR correlations (

n{=}20

target-model

\times

task-type cells,

p<0.0001

for all).

Judge	Target Model	TI	IE	IC	MT	TT	IT	MU
Human avg	Gemini-2.5	1.42	1.67	1.74	1.77	1.69	1.27	1.71
	GPT-5	0.92	0.97	1.47	1.03	0.63	0.54	1.11
	Qwen	2.14	1.92	—	1.87	1.96	1.93	2.05
Gemini-2.5 Pro	Gemini-2.5	1.31	1.57	1.55	1.71	1.18	0.85	1.22
	GPT-5	0.80	0.87	1.32	0.99	0.35	0.34	0.66
	Qwen	2.08	1.81	—	1.69	1.60	1.60	1.68

Appendix

Table 6: Per-cell ARR by judge model across target models and task types. “—” denotes unsupported task combinations.

UMM	Image output								Text output
	TI		IE		IC		MT		TT		IT		MU
	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR
GPT-5 [ 52 ]	27.3	0.76	31.6	0.94	34.7	1.19	34.1	0.95	2.1	0.41	2.9	0.37	5.1	0.70
Gemini-2.5 [ 19 ]	45.4	1.30	50.7	1.63	38.7	1.45	57.0	1.69	22.6	1.08	17.8	0.85	25.2	1.23
Qwen-Image [ 82 ]	69.6	2.07	52.3	1.74	-	-	45.5	1.62	-	-	-	-	-	-
Qwen2.5-VL [ 5 ]	-	-	-	-	-	-	-	-	39.6	1.53	38.1	1.49	36.5	1.57

Appendix

Table 7: Individual judge safety evaluation results: Gemini-2.5 Pro . Format follows Table 2 . Bold indicates the highest value per column.

UMM	Image output								Text output
	TI		IE		IC		MT		TT		IT		MU
	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR
GPT-5 [ 52 ]	28.5	0.86	32.1	1.11	42.6	1.52	31.3	1.06	12.3	0.58	9.9	0.50	23.5	1.02
Gemini-2.5 [ 19 ]	48.9	1.48	59.0	1.97	54.4	1.94	58.3	1.96	52.3	1.74	38.4	1.32	49.9	1.74
Qwen-Image [ 82 ]	79.9	2.39	66.5	2.25	-	-	61.4	2.14	-	-	-	-	-	-
Qwen2.5-VL [ 5 ]	-	-	-	-	-	-	-	-	65.2	2.14	63.1	2.08	59.3	2.09

Appendix

Table 8: Individual judge safety evaluation results: GPT-5 Nano . Format follows Table 2 . Bold indicates the highest value per column.

UMM	Image output								Text output
	TI		IE		IC		MT		TT		IT		MU
	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR	ASR	ARR
GPT-5 [ 52 ]	23.4	0.72	23.3	0.83	27.2	1.10	28.5	0.89	4.2	0.43	3.8	0.38	9.5	0.73
Gemini-2.5 [ 19 ]	37.7	1.18	45.0	1.56	37.4	1.47	46.4	1.56	34.3	1.24	23.3	0.92	32.9	1.29
Qwen-Image [ 82 ]	69.8	2.07	53.3	1.78	-	-	54.9	1.85	-	-	-	-	-	-
Qwen2.5-VL [ 5 ]	-	-	-	-	-	-	-	-	41.2	1.49	40.8	1.46	34.8	1.47

Appendix

Table 9: Individual judge safety evaluation results: Qwen-2.5 VL-72B . Format follows Table 2 . Bold indicates the highest value per column.

Panel of Figure 10 — Figure 10: Category-wise ASR Heatmaps for different models: GPT-5, Gemini-2.5, and OmniGen-2.

Task	Global	GPT-5	Gemini-2.5	OmniGen-2
Text-to-Image	0.1755	0.2931	0.1595	0.1716
Image Editing	0.0869	0.1313	0.1782	0.1260
Composition	0.1857	0.1451	0.1526	0.3371
Multi-turn	0.1197	0.1819	0.1757	0.4683
Text-to-Text	0.3310	0.3913	0.3333	0.3515
Image Captioning	0.3521	0.4047	0.3597	0.3827

Appendix

Table 10: We report the safety fluctuation defined by standard deviation divided by mean ASR for different categories. We report the values of 3 models: GPT-5, Gemini-2.5, OmniGen-2, and average values of the 3 models. Highest value among different tasks is bolded for each model.

Panel of Figure 11 — Figure 11: Average safety scores (ARR, ASR) of open sourced models and commercial models across all tasks: (a) average ARR(Average Risk Rating) of the commercial models and open source models (b) average ASR(Attack Success Rate) of the commercial models and open source models.

Figure 12: Conditional rate of high-risk samples (risk rating 3) among non-refused samples. Average values are plotted for open-source models and commercial models.

Figure 13: Average ARR for image-output tasks and text-output tasks for different UMMs.

Figure 14: Average ASR for image-output tasks and text-output tasks for different UMMs.

Panel of Figure 25 — (a) Prompt for generating minimal unsafe triggers for Image Output tasks.

Panel of Figure 26 — (a) Prompt for generating minimal unsafe triggers for Text Output tasks.

Panel of Figure 17 — Figure 17: Prompts used in the Unsafe Trigger Construction pipeline. We utilize separate prompts for (a) Image and (b) Text modalities for AI-assisted bulk generation. Additionally, (c) a selection prompt is provided, which enables the full automation of the filtering process aligned with our safety taxonomy, and served as a baseline reference for our human curation.

Panel of Figure 18 — Figure 18: Prompts used for expanding atomic unsafe triggers into full target scenarios (Appendix C.5 ). (a) expands image triggers into coherent scenes, while (b) wraps text triggers in benign-sounding operational instructions.

Panel of Figure 29 — (a) Prompt for Image Editing (IE). The model decomposes the target into a safe base and a context-dependent instruction.

Panel of Figure 30 — (a) Prompt for Image Composition (IC). The unsafe concept is split across two benign images and reassembled via instruction.

Panel of Figure 21 — Figure 21: Prompts used for Scenario Generation on Image Tasks.

Panel of Figure 22 — Figure 22: Prompts for Multimodal Understanding (MU) scenarios. The Cross-Modal Trigger Split strategy pairs harmful images with benign text instructions.

Panel of Figure 23 — Figure 23: The risk rating guidelines used for the Image Tasks evaluation prompt.

Panel of Figure 24 — Figure 24: The risk rating guidelines used for the Text Tasks evaluation prompt.

Panel of Figure 35 — (a) Text-to-Image (TI)

Panel of Figure 36 — (a) Image Editing (IE)

Panel of Figure 37 — (a) Image Composition (IC)

Panel of Figure 28 — Figure 28: Evaluation prompts for the four Image Tasks: (a) Text-to-Image, (b) Image Editing, (c) Image Composition, and (d) Multi-turn.

Panel of Figure 31 — Figure 31: Safety evaluation instructions provided to human evaluators.

Panel of Figure 32 — Figure 32: Example of input data from the Violence (V2) category in the UniSAFE image outputs dataset.

Panel of Figure 33 — Figure 33: Example of input data from the Hate (H1) category in the UniSAFE image outputs dataset.

Panel of Figure 34 — Figure 34: Example of input data from the Sexual (S1) category in the UniSAFE image outputs dataset.

Unified Multimodal Models

Unified multimodal models combine multiple data types in shared systems for cross-modal understanding, generation, reasoning, and interaction.

See all 282 papers

Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models
Apr 1, 2026Zixiang Peng, Yongxiu Xu, Qin-Yi Zhang +5Unified Multimodal Models LLM Safety Benchmarks
Institute of Information Engineering, Chinese Academy of Sciences School of Cyber Security, University of Chinese Academy of Sciences Beijing, China · Institute of Automation, Chinese Academy of Sciences School of Artificial Intelligence, University of Chinese Academy of Sciences Beijing, China
Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Models
Mar 28, 2026Kaishen Wang, Heng HuangMultimodal Robustness Unified Multimodal Models
University of Maryland, College Park, MD 20742, USA
UnifiedAttack: Evaluating the Safety of Large Multimodal Models in Synergistic Harmful Image-Text Generation
Sep 29, 2026Bingjun Luo, Jialin Guo, Tony Wang +1Multimodal Jailbreak Attacks Adversarial Attacks on Unified Multimodal Models
Tsinghua University · Harbin Engineering University

Apr 1, 2026J/K move · Enter open · S save

Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models

Zixiang Peng, Yongxiu Xu, Qin-Yi Zhang, Jiexun Shen+4

Institute of Information Engineering, Chinese Academy of Sciences School of Cyber Security, University of Chinese Academy of Sciences Beijing, China · Institute of Automation, Chinese Academy of Sciences School of Artificial Intelligence, University of Chinese Academy of Sciences Beijing, China

Unified Multimodal Models LLM Safety Benchmarks LLM Safety Alignment Multimodal Model Evaluation

Open paper AskPDF arXiv

Unified Multimodal Large Models (UMLMs) integrate understanding and generation capabilities within a single architecture. While unified architectures expand multimodal capabilities, their safety implications remain important yet underexplored. Existing safety benchmarks predominantly focus on isolated understanding or generation tasks, failing to evaluate the holistic safety of UMLMs when handling diverse tasks under a unified framework. To address this, we introduce Uni-SafeBench, a comprehensive benchmark featuring a taxonomy of six major safety categories across seven task types. To ensure rigorous assessment, we develop Uni-Judger, a framework that effectively decouples contextual safety from intrinsic safety. Based on comprehensive evaluations across Uni-SafeBench, we find that the original safety alignment of the underlying LLM is not consistently preserved in current unified models. Moreover, open-source UMLMs exhibit much lower safety performance than multimodal large models specialized for either generation or understanding tasks, particularly on the generation side.

Abstract

Unified Multimodal Models

Explore similar work

Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models

Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Models

UnifiedAttack: Evaluating the Safety of Large Multimodal Models in Synergistic Harmful Image-Text Generation