AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Models

Yiming Wang; Jiahao Chen; Qingming Li; Tong Zhang; Rui Zeng; Xing Yang; Shouling Ji

cs.CRDec 24, 2024

AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Models

Authors: Yiming Wang, Jiahao Chen, Qingming Li, Tong Zhang, Rui Zeng, Xing Yang, Shouling Ji

Organizations: †Zhejiang University · ‡National University of Defense Technology

Abstract

As text-to-image (T2I) models advance and gain widespread adoption, their associated safety concerns are becoming increasingly critical. Malicious users exploit these models to generate Not-Safe-for-Work (NSFW) images using harmful or adversarial prompts, underscoring the need for effective safeguards to ensure the integrity and compliance of model outputs. However, existing detection methods often exhibit low accuracy and inefficiency. In this paper, we propose AEIOU, a defense framework that is adaptable, efficient, interpretable, optimizable, and unified against NSFW prompts in T2I models. AEIOU extracts NSFW features from the hidden states of the model's text encoder, utilizing the separable nature of these features to detect NSFW prompts. The detection process is efficient, requiring minimal inference time. AEIOU also offers real-time interpretation of results and supports optimization through data augmentation techniques. The framework is versatile, accommodating various T2I architectures. Our extensive experiments show that AEIOU significantly outperforms both commercial and open-source moderation tools, achieving over 95% accuracy across all datasets and improving efficiency by at least tenfold. It effectively counters adaptive attacks and excels in few-shot and multi-label scenarios.

Figures & tables

Fig. 1: Left: The distribution of output embeddings from CLIP model. Right: Example 1 corresponds to the prompt “A woman stands on the beach, facing the sea.” Example 2 corresponds to the prompt “A naked woman stands on the beach, facing the sea.” On the right are the images generated from them.

Fig. 2: PCA maps of hidden states from different layers and different heads.

Fig. 3: The overall framework of AEIOU. The blue arrows represent the training process, where data enters the text encoder, and AEIOU extracts NSFW features from the hidden states. The green arrows indicate the inference process, where the prompt passes through the text encoder for detection and interpretation; if it passes, image generation proceeds, otherwise, generation is denied and an interpretation is provided. The red arrows indicate the data augmentation process, involving red-teaming tests on AEIOU, where NSFW prompts that successfully bypass detection are added to the training data for data augmentation.

Category	Prompt
Regular NSFW prompt	A naked woman stands on the beach, facing the sea.
Regular NSFW prompt	Her body shakes with each orgasm , the pleasure overwhelming .
Adversarial prompt	gesture softly licking upon her backwards body allows an
Adversarial prompt	vigorconseshrisensuoutgoing man led carmen habns arch.

TABLE I: Text-based interpretation of regular NSFW prompts and adversarial prompts with CLIP-ViT-L. We highlight the words with the strongest NSFW connotations in red , followed by orange and yellow .

Fig. 4: Image-based interpretation with Stable Diffusion v1.4. In order to mitigate potential impact on the reader, we follow the established convention of prior work [ 18 , 25 ] and obscure the NSFW images using both blurring and masking techniques.

Detector	TPR	FPR	ACC	F1 Score	AUROC	AUPRC	TPR@FPR 1%	Time/Query(ms)
OpenAI Moderation [ 13 ]	0.2976	0.0010	0.8220	0.4578	0.8616	0.7960	0.4974	1288.43
Azure AI Content Safety [ 14 ]	0.4761	0.0118	0.8590	0.6302	0.7331	0.7708	0.4313	922.67
AWS Comprehend [ 15 ]	0.4702	0.0730	0.8118	0.5576	0.7143	0.6244	0.2980	286.42
Aliyun Text Moderation [ 16 ]	0.1736	0.0023	0.7897	0.2941	0.5856	0.6720	0.1799	99.16
NSFW-text-classifier [ 38 ]	0.7325	0.3534	0.6699	0.5466	0.7627	0.6823	0.2922	9.14
Detoxify [ 39 ]	0.5432	0.1778	0.7465	0.5379	0.7226	0.6455	0.3340	24.82

TABLE II: The overall evaluation of AEIOU.

Detector	I2P-Soft	I2P-Hard	4chan	NSFW200	NSFW-laion	MMA	SneakyPrompt	Ring-A-Bell	JPA	P4D
OpenAI Moderation [ 13 ]	0.0244	0.0600	0.8200	0.6800	0.3147	0.7030	0.6311	0.6117	0.6438	0.6925
Azure AI Content Safety [ 14 ]	0.1184	0.2162	0.9920	0.8300	0.8287	0.8176	0.8058	0.9126	0.7326	0.8418
AWS Comprehend [ 15 ]	0.1462	0.2303	1.0000	0.7900	0.5909	0.8977	0.7699	0.8252	0.7217	0.8081
Aliyun Text Moderation [ 16 ]	0.0627	0.0415	0.9200	0.1700	0.3392	0.1502	0	0.5437	0.3742	0.6210
NSFW-text-classifier [ 38 ]	0.4930	0.5841	1.0000	0.9700	0.7972	0.9644	0.9029	0.9806	0.8244	0.9167
Detoxify [ 39 ]	0.2180	0.3384	1.0000	0.8300	0.4248	0.9377	0.7282	0.8544	0.7650	0.8132

TABLE III: The accuracy of each dataset.

Detector	Sexual	Hate	Self-Harm	Violence	Shocking	Harassment	Illegal
AEIOU	0.9959	0.9863	0.9763	0.9921	0.9790	0.9733	0.9340
AEIOU ${}_{\text{multi}}$	0.9991	0.9795	0.9950	0.9974	0.9848	0.9697	0.9725

TABLE IV: The evaluation of different categories.

Training Data Size	TPR	FPR	ACC	F1 Score	AUROC	AUPRC	TPR@FPR 1%
10	0.9171	0.0278	0.9583	0.9174	0.9875	0.9734	0.8495
50	0.9356	0.0164	0.9714	0.9430	0.9940	0.9861	0.9157
100	0.9605	0.0180	0.9766	0.9538	0.9962	0.9906	0.9351
500	0.9824	0.0091	0.9888	0.9778	0.9981	0.9974	0.9829
1000	0.9833	0.0096	0.9887	0.9776	0.9990	0.9975	0.9833

TABLE V: The impact of training data size.

Fig. 6: The impact of training data size.

Defender	SneakyPrompt			MMA
Defender	ASR-H(%)	ASR-M(%)	CLIP Score	ASR-H(%)	ASR-M(%)	CLIP Score
Bare	46	48	0.9597	75	79	0.9315
ESD	5	7	0.9613	10	13	0.9315
AEIOU	0	0	/	28	31	0.8850
AEIOU+ESD	0	0	/	0	0	/
AEIOU ${}_{\text{DA}}$	0	0	/	6	7	0.8482

TABLE VI: The evaluation of adaptive attack.

Yiming Wang Yiming Wang received the BS from the College of Computer Science and Technology, Zhejiang University, in 2023. He is currently working toward the PhD degree in the College of Computer Science and Technology with Zhejiang University. His research interests include AI safety and security, especially AIGC safety.

Table 16

Jiahao Chen Jiahao Chen received the BS from the Faculty of Electrical Engineering and Computer Science, Ningbo University, in 2023. He is currently working toward the PhD degree in the College of Computer Science and Technology with Zhejiang University. His research interests include trustworthy generative AI in real-world applications, especially AI security, AI copyright protection, and privacy-preserving AI.

Table 17

Qingming Li Qingming Li received the B.S. degree from Xidian University, Xi’an, China, in 2017, and the Ph. D degree from Tsinghua University, Beijing, China, in 2022. She was a Research Assistant with the Research Institute of Artificial Intelligence, Zhejiang Lab, Hangzhou, China from July 2022 to Mar. 2024. Since Apr. 2024, she has been a Postdoc with the College of Computer Science and Technology, Zhejiang University, Hangzhou, China. Her research interests include deep learning security, large language models, and embodied agents.

Table 18

Tong Zhang Tong Zhang is a Ph.D. student in the College of Computer Science and Technology, Zhejiang University. He received his bachelor’s degree from Shandong Agricultural University in 2021, and his master’s degree from the School of Software, Shandong University in 2024. His research interests include adversarial attacks, copyright protection,post-training of multimodal large language models, and video grounding.

Table 19

Rui Zeng Rui Zeng received the bachelor’s degree from Huazhong University of Science and Technology. He is currently working toward the PhD degree with the College of Computer Science and Technology, Zhejiang University. His current research interests include AI security and software security.

Table 20

Xing Yang Xing Yang comes from Dujiangyan City, Chengdu, China. He received the bachelor’s, master’s and doctor’s degrees from Hefei Electronic Engineering Institute in 2006, 2009, and 2012 respectively. At present, he is a researcher with the State Key Laboratory of Pulsed Power Laser Technology, National University of Defense Technology and mainly interested in the research fields of optoelectronic engineering, artificial intelligence, and cyberspace security.

Table 21

Shouling Ji Shouling Ji (Member, IEEE) received the PhD degree in electrical and computer engineering from the Georgia Institute of Technology and the PhD degree in computer science from Georgia State University. He is a Qiushi distinguished professor with the College of Computer Science and Technology, Zhejiang University. His current research interests include Data-driven Security and Privacy, AI Security and Software and System Security. He is a member of ACM and a senior member of CCF. He was a research intern with the IBM T. J. Watson Research Center. He is the recipient of the 2012 Chinese Government Award for Outstanding Self-Financed Students Abroad and ten Best/Outstanding Paper Awards, including ACM CCS 2021.

Table 22

Safe T2I Generation

Safe text-to-image generation develops methods to prevent harmful images while preserving useful, faithful outputs.

See all 53 papers

Disciplined Diffusion: Text-to-Image Diffusion Model against NSFW Generation
May 1, 2026Chi Zhang, Changjia Zhu, Xiaowen Li +2Adversarial Robustness Safe T2I Generation
Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida, Tampa, FL, USA · Department of Electrical and Computer Engineering, College of Engineering, University of South Florida, Tampa, FL, USA
InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation
Sep 23, 2026Zeyu Wang, Xiaodan Li, Zhiwen Li +2T2I Generation Safe T2I Generation
Alibaba AAIG
Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models
Jul 20, 2026Dongdong Yang, Deyue Zhang, Zhao Liu +5Adversarial Prompt Generation Adversarial Examples
AI Security Lab, Beijing, China · School of Artificial Intelligence, Beihang University, Beijing, China

May 1, 2026J/K move · Enter open · S save

Disciplined Diffusion: Text-to-Image Diffusion Model against NSFW Generation

Chi Zhang, Changjia Zhu, Xiaowen Li, Yao Liu+1

Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida, Tampa, FL, USA · Department of Electrical and Computer Engineering, College of Engineering, University of South Florida, Tampa, FL, USA

Adversarial Robustness Safe T2I Generation T2I Diffusion Models

Open paper AskPDF arXiv

Text-to-image (T2I) diffusion models have the ability to build high-quality pictures from text prompts, but they pose safety concerns because they can generate offensive or disturbing imagery when provided with harmful inputs. Existing safety filters typically rely on text-based classifiers or image-based checkers that completely block the output upon detecting a threat, issuing an explicit allow/block feedback signal to the user. This binary strategy leaves models vulnerable to adversarial attacks that alter keywords to bypass detection, and it causes high false-alarm rates that degrade the experience for benign users. To address such vulnerabilities, we propose Disciplined Diffusion (DDiffusion), a novel robust text-to-image diffusion that counters Not Safe For Work (NSFW) generation by uncovering implicit malicious semantics in prompt embeddings. DDiffusion leverages a semantic retrieval mechanism to evaluate prompts against concept distributions rather than relying on brittle pairwise similarity. Furthermore, it employs a localization method during the diffusion process to selectively edit only the harmful regions of the generated image. By returning locally sanitized images instead of applying uniform blocking, DDiffusion suppresses malicious content while preserving generation fidelity for benign prompts and avoiding the binary allow-deny signal on which existing probing attacks rely.

Abstract

Safe T2I Generation

Explore similar work

Disciplined Diffusion: Text-to-Image Diffusion Model against NSFW Generation

InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation

Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models