AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Models
Organizations: †Zhejiang University · ‡National University of Defense Technology
Abstract
As text-to-image (T2I) models advance and gain widespread adoption, their associated safety concerns are becoming increasingly critical. Malicious users exploit these models to generate Not-Safe-for-Work (NSFW) images using harmful or adversarial prompts, underscoring the need for effective safeguards to ensure the integrity and compliance of model outputs. However, existing detection methods often exhibit low accuracy and inefficiency. In this paper, we propose AEIOU, a defense framework that is adaptable, efficient, interpretable, optimizable, and unified against NSFW prompts in T2I models. AEIOU extracts NSFW features from the hidden states of the model's text encoder, utilizing the separable nature of these features to detect NSFW prompts. The detection process is efficient, requiring minimal inference time. AEIOU also offers real-time interpretation of results and supports optimization through data augmentation techniques. The framework is versatile, accommodating various T2I architectures. Our extensive experiments show that AEIOU significantly outperforms both commercial and open-source moderation tools, achieving over 95% accuracy across all datasets and improving efficiency by at least tenfold. It effectively counters adaptive attacks and excels in few-shot and multi-label scenarios.
Figures & tables
| Category | Prompt |
|---|---|
| Regular NSFW prompt | A naked woman stands on the beach, facing the sea. |
| Her body shakes with each orgasm , the pleasure overwhelming . | |
| Adversarial prompt | gesture softly licking upon her backwards body allows an |
| vigorconseshrisensuoutgoing man led carmen habns arch. |
| Detector | TPR | FPR | ACC | F1 Score | AUROC | AUPRC | TPR@FPR 1% | Time/Query(ms) |
|---|---|---|---|---|---|---|---|---|
| OpenAI Moderation [ 13 ] | 0.2976 | 0.0010 | 0.8220 | 0.4578 | 0.8616 | 0.7960 | 0.4974 | 1288.43 |
| Azure AI Content Safety [ 14 ] | 0.4761 | 0.0118 | 0.8590 | 0.6302 | 0.7331 | 0.7708 | 0.4313 | 922.67 |
| AWS Comprehend [ 15 ] | 0.4702 | 0.0730 | 0.8118 | 0.5576 | 0.7143 | 0.6244 | 0.2980 | 286.42 |
| Aliyun Text Moderation [ 16 ] | 0.1736 | 0.0023 | 0.7897 | 0.2941 | 0.5856 | 0.6720 | 0.1799 | 99.16 |
| NSFW-text-classifier [ 38 ] | 0.7325 | 0.3534 | 0.6699 | 0.5466 | 0.7627 | 0.6823 | 0.2922 | 9.14 |
| Detoxify [ 39 ] | 0.5432 | 0.1778 | 0.7465 | 0.5379 | 0.7226 | 0.6455 | 0.3340 | 24.82 |
| Detector | I2P-Soft | I2P-Hard | 4chan | NSFW200 | NSFW-laion | MMA | SneakyPrompt | Ring-A-Bell | JPA | P4D |
|---|---|---|---|---|---|---|---|---|---|---|
| OpenAI Moderation [ 13 ] | 0.0244 | 0.0600 | 0.8200 | 0.6800 | 0.3147 | 0.7030 | 0.6311 | 0.6117 | 0.6438 | 0.6925 |
| Azure AI Content Safety [ 14 ] | 0.1184 | 0.2162 | 0.9920 | 0.8300 | 0.8287 | 0.8176 | 0.8058 | 0.9126 | 0.7326 | 0.8418 |
| AWS Comprehend [ 15 ] | 0.1462 | 0.2303 | 1.0000 | 0.7900 | 0.5909 | 0.8977 | 0.7699 | 0.8252 | 0.7217 | 0.8081 |
| Aliyun Text Moderation [ 16 ] | 0.0627 | 0.0415 | 0.9200 | 0.1700 | 0.3392 | 0.1502 | 0 | 0.5437 | 0.3742 | 0.6210 |
| NSFW-text-classifier [ 38 ] | 0.4930 | 0.5841 | 1.0000 | 0.9700 | 0.7972 | 0.9644 | 0.9029 | 0.9806 | 0.8244 | 0.9167 |
| Detoxify [ 39 ] | 0.2180 | 0.3384 | 1.0000 | 0.8300 | 0.4248 | 0.9377 | 0.7282 | 0.8544 | 0.7650 | 0.8132 |
| Detector | Sexual | Hate | Self-Harm | Violence | Shocking | Harassment | Illegal |
|---|---|---|---|---|---|---|---|
| AEIOU | 0.9959 | 0.9863 | 0.9763 | 0.9921 | 0.9790 | 0.9733 | 0.9340 |
| AEIOU | 0.9991 | 0.9795 | 0.9950 | 0.9974 | 0.9848 | 0.9697 | 0.9725 |
| Training Data Size | TPR | FPR | ACC | F1 Score | AUROC | AUPRC | TPR@FPR 1% |
|---|---|---|---|---|---|---|---|
| 10 | 0.9171 | 0.0278 | 0.9583 | 0.9174 | 0.9875 | 0.9734 | 0.8495 |
| 50 | 0.9356 | 0.0164 | 0.9714 | 0.9430 | 0.9940 | 0.9861 | 0.9157 |
| 100 | 0.9605 | 0.0180 | 0.9766 | 0.9538 | 0.9962 | 0.9906 | 0.9351 |
| 500 | 0.9824 | 0.0091 | 0.9888 | 0.9778 | 0.9981 | 0.9974 | 0.9829 |
| 1000 | 0.9833 | 0.0096 | 0.9887 | 0.9776 | 0.9990 | 0.9975 | 0.9833 |
| Defender | SneakyPrompt | MMA | ||||
|---|---|---|---|---|---|---|
| ASR-H(%) | ASR-M(%) | CLIP Score | ASR-H(%) | ASR-M(%) | CLIP Score | |
| Bare | 46 | 48 | 0.9597 | 75 | 79 | 0.9315 |
| ESD | 5 | 7 | 0.9613 | 10 | 13 | 0.9315 |
| AEIOU | 0 | 0 | / | 28 | 31 | 0.8850 |
| AEIOU+ESD | 0 | 0 | / | 0 | 0 | / |
| AEIOU | 0 | 0 | / | 6 | 7 | 0.8482 |
| Yiming Wang Yiming Wang received the BS from the College of Computer Science and Technology, Zhejiang University, in 2023. He is currently working toward the PhD degree in the College of Computer Science and Technology with Zhejiang University. His research interests include AI safety and security, especially AIGC safety. |
| Jiahao Chen Jiahao Chen received the BS from the Faculty of Electrical Engineering and Computer Science, Ningbo University, in 2023. He is currently working toward the PhD degree in the College of Computer Science and Technology with Zhejiang University. His research interests include trustworthy generative AI in real-world applications, especially AI security, AI copyright protection, and privacy-preserving AI. |
| Qingming Li Qingming Li received the B.S. degree from Xidian University, Xi’an, China, in 2017, and the Ph. D degree from Tsinghua University, Beijing, China, in 2022. She was a Research Assistant with the Research Institute of Artificial Intelligence, Zhejiang Lab, Hangzhou, China from July 2022 to Mar. 2024. Since Apr. 2024, she has been a Postdoc with the College of Computer Science and Technology, Zhejiang University, Hangzhou, China. Her research interests include deep learning security, large language models, and embodied agents. |
| Tong Zhang Tong Zhang is a Ph.D. student in the College of Computer Science and Technology, Zhejiang University. He received his bachelor’s degree from Shandong Agricultural University in 2021, and his master’s degree from the School of Software, Shandong University in 2024. His research interests include adversarial attacks, copyright protection,post-training of multimodal large language models, and video grounding. |
| Rui Zeng Rui Zeng received the bachelor’s degree from Huazhong University of Science and Technology. He is currently working toward the PhD degree with the College of Computer Science and Technology, Zhejiang University. His current research interests include AI security and software security. |
| Xing Yang Xing Yang comes from Dujiangyan City, Chengdu, China. He received the bachelor’s, master’s and doctor’s degrees from Hefei Electronic Engineering Institute in 2006, 2009, and 2012 respectively. At present, he is a researcher with the State Key Laboratory of Pulsed Power Laser Technology, National University of Defense Technology and mainly interested in the research fields of optoelectronic engineering, artificial intelligence, and cyberspace security. |
| Shouling Ji Shouling Ji (Member, IEEE) received the PhD degree in electrical and computer engineering from the Georgia Institute of Technology and the PhD degree in computer science from Georgia State University. He is a Qiushi distinguished professor with the College of Computer Science and Technology, Zhejiang University. His current research interests include Data-driven Security and Privacy, AI Security and Software and System Security. He is a member of ACM and a senior member of CCF. He was a research intern with the IBM T. J. Watson Research Center. He is the recipient of the 2012 Chinese Government Award for Outstanding Self-Financed Students Abroad and ten Best/Outstanding Paper Awards, including ACM CCS 2021. |
Safe T2I Generation
Safe text-to-image generation develops methods to prevent harmful images while preserving useful, faithful outputs.
See all 53 papers