Safe Image Generation via Reinforcement Learning

Eungyeol Han; Jong-Seok Lee

cs.CVOct 5, 2026

Safe Image Generation via Reinforcement Learning

Authors: Eungyeol Han, Jong-Seok Lee

Organizations: Yonsei University School of Integrated Technology

Abstract

Recent Text-to-Image (T2I) models achieve remarkable visual image generation performance, but they can still generate NSFW (Not-Safe-For-Work) contents, including violent or explicit images. Existing safety checker mechanisms are largely confined to pre-generation filtering (e.g. prompt-level text classifiers) or post-hoc moderation applied after an image is completely synthesized. However, adversarial attack methods operate over a much broader space. This imbalance highlights the need for a safety mechanism that intervenes during the generation process. We propose an in-generation safety framework that monitors the denoising trajectory and detects emerging NSFW signals from intermediate representations. Rather than merely detecting NSFW generations, our method applies reinforcement learning to generate safe images from NSFW prompts. By coupling in-generation detection with controllable steering, our approach mitigates unsafe trajectories even when NSFW signals emerge after generation has already begun. Experiments results show that our method consistently outperforms existing safe image generation methods across both standard and adversarial evaluation sets, while preserving perceptual quality and prompt fidelity. Code will be released upon acceptance.

Figures & tables

Figure 1: Overall pipeline of the proposed trajectory-level safety framework.

Method	Inappropriate Probability $\downarrow$						Exp. Max. Inappropriateness $\downarrow$
	SD v1.5		SD v2.0		SD v3.0		SD v1.5		SD v2.0		SD v3.0
	Naive	Adv	Naive	Adv	Naive	Adv	Naive	Adv	Naive	Adv	Naive	Adv
SD Vanilla	0.41	0.49	0.42	0.47	0.44	0.52	$0.95^{\pm 0.05}$	$0.96^{\pm 0.01}$	$0.94^{\pm 0.01}$	$0.98^{\pm 0.01}$	$0.97^{\pm 0.02}$	$0.98^{\pm 0.01}$
SLD-Max [ 38 ]	0.23	0.20	0.22	0.21	0.24	0.25	$0.71^{\pm 0.19}$	$0.72^{\pm 0.06}$	$0.73^{\pm 0.24}$	$0.74^{\pm 0.15}$	$0.74^{\pm 0.02}$	$0.77^{\pm 0.2}$
SAFREE [ 47 ]	0.21	0.20	0.20	0.22	0.23	0.23	$0.69^{\pm 0.25}$	$0.70^{\pm 0.23}$	$0.72^{\pm 0.24}$	$0.75^{\pm 0.23}$	$0.75^{\pm 0.12}$	$0.76^{\pm 0.14}$
PurifyGen [ 3 ]	0.22	0.21	0.23	0.22	-	-	$0.70^{\pm 0.14}$	$0.71^{\pm 0.21}$	$0.74^{\pm 0.15}$	$0.73^{\pm 0.14}$	-	-

Table 1: Quantitative comparison of our method against existing methods across T2I models and evaluation sets in terms of Inappropriate Probability and Expected Maximum Inappropriateness (std). PurifyGen, UCE, and RECE are not evaluated on SD v3.0, as their official implementations are not applicable to SD v3.0

Figure 2: Generated examples of ours and other safe image generation methods.

Method	SD v1.5			SD v2.0			SD v3.0
Method	FID $\downarrow$	LPIPS $\downarrow$	CLIP $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	CLIP $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	CLIP $\uparrow$
SD Vanilla	14.43	0.05	0.33	14.12	0.05	0.34	13.97	0.04	0.32
SLD-Max [ 38 ]	18.79	0.17	0.29	17.99	0.17	0.30	17.44	0.19	0.29
SAFREE [ 47 ]	17.12	0.13	0.29	17.01	0.12	0.28	16.99	0.12	0.29
PurifyGen [ 3 ]	17.75	0.10	0.27	17.64	0.10	0.28	–	–	–
UCE [ 11 ]	16.44	0.11	0.26	16.41	0.10	0.26	–	–	–

Table 2: Image quality comparison on SD v1.5, SD v2.0, and SD v3.0. Bold and underlined values indicate the best and second-best results, respectively.

Table 5

Method	IP $\downarrow$		EMI $\downarrow$		FID $\downarrow$	CLIP $\uparrow$
Method	Naive	Adv	Naive	Adv	FID $\downarrow$	CLIP $\uparrow$
w/o intermediate reward	0.31	0.29	$0.82^{\pm 0.01}$	$0.83^{\pm 0.04}$	15.92	0.29
w/o terminal reward	0.22	0.19	$0.71^{\pm 0.19}$	$0.72^{\pm 0.07}$	18.43	0.27
fixed $a_{t}=0.5$	0.26	0.24	$0.76^{\pm 0.12}$	$0.77^{\pm 0.09}$	17.21	0.28
w/o masking	0.23	0.20	$0.72^{\pm 0.10}$	$0.73^{\pm 0.11}$	17.84	0.27
Memoryless MLP	0.22	0.20	$0.71^{\pm 0.12}$	$0.72^{\pm 0.04}$	16.12	0.29

Table 5: Ablation study on SD v1.5. IP: Inappropriate Probability, EMI: Exp. Max. Inappropriateness.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Hate		Harassment		Violence		Self-harm		Sexual		Shocking		Illegal act.		Overall
Method	N	A	N	A	N	A	N	A	N	A	N	A	N	A	N	A
SD Vanilla	0.41	0.55	0.36	0.48	0.45	0.50	0.40	0.46	0.44	0.44	0.54	0.60	0.33	0.41	0.41	0.49
SLD-Max	0.27	0.22	0.19	0.20	0.25	0.26	0.20	0.18	0.22	0.14	0.29	0.30	0.17	0.16	0.23	0.20
SAFREE	0.25	0.21	0.18	0.18	0.24	0.24	0.19	0.17	0.21	0.13	0.27	0.28	0.16	0.15	0.21	0.20
PurifyGen	0.24	0.20	0.19	0.17	0.23	0.22	0.20	0.17	0.22	0.14	0.28	0.27	0.17	0.16	0.22	0.21
UCE	0.26	0.33	0.20	0.27	0.25	0.32	0.21	0.27	0.23	0.30	0.29	0.36	0.18	0.24	0.23	0.30

Appendix

Table 6: Per-category Inappropriate Probability (

\downarrow

) on SD v1.5. N: Naive, A: Adversarial.

Method	Hate		Harassment		Violence		Self-harm		Sexual		Shocking		Illegal act.		Overall
Method	N	A	N	A	N	A	N	A	N	A	N	A	N	A	N	A
SD Vanilla	0.39	0.52	0.35	0.46	0.43	0.49	0.38	0.44	0.42	0.43	0.52	0.58	0.31	0.40	0.42	0.47
SLD-Max	0.25	0.21	0.18	0.19	0.24	0.25	0.19	0.17	0.21	0.13	0.28	0.29	0.16	0.15	0.22	0.21
SAFREE	0.23	0.20	0.17	0.17	0.23	0.23	0.18	0.16	0.20	0.12	0.26	0.27	0.15	0.14	0.20	0.22
PurifyGen	0.23	0.19	0.18	0.16	0.22	0.21	0.19	0.16	0.21	0.13	0.27	0.26	0.16	0.15	0.23	0.22
UCE	0.25	0.31	0.19	0.26	0.24	0.30	0.20	0.26	0.22	0.29	0.28	0.35	0.17	0.23	0.23	0.27

Appendix

Table 7: Per-category Inappropriate Probability (

\downarrow

) on SD v2.0. N: Naive, A: Adversarial.

Method	Hate		Harassment		Violence		Self-harm		Sexual		Shocking		Illegal act.		Overall
Method	N	A	N	A	N	A	N	A	N	A	N	A	N	A	N	A
SD Vanilla	0.43	0.57	0.39	0.50	0.47	0.53	0.41	0.48	0.45	0.46	0.56	0.62	0.34	0.43	0.44	0.52
SLD-Max	0.26	0.23	0.20	0.21	0.26	0.27	0.21	0.19	0.23	0.15	0.30	0.31	0.18	0.17	0.24	0.25
SAFREE	0.24	0.22	0.19	0.19	0.25	0.25	0.20	0.18	0.22	0.14	0.28	0.29	0.17	0.16	0.23	0.23
PurifyGen	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–
UCE	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–

Appendix

Table 8: Per-category Inappropriate Probability (

\downarrow

) on SD v3.0. N: Naive, A: Adversarial. ‘–’ indicates method not applicable.

Method	Hate		Harassment		Violence		Self-harm		Sexual		Shocking		Illegal act.		Overall
Method	N	A	N	A	N	A	N	A	N	A	N	A	N	A	N	A
SD Vanilla	$0.96^{\pm.03}$	$0.98^{\pm.01}$	$0.93^{\pm.02}$	$0.95^{\pm.05}$	$0.91^{\pm.09}$	$0.92^{\pm.07}$	$0.98^{\pm.02}$	$0.98^{\pm.01}$	$0.93^{\pm.07}$	$0.93^{\pm.05}$	$0.98^{\pm.01}$	$0.97^{\pm.03}$	$0.95^{\pm.04}$	$0.94^{\pm.04}$	$0.95^{\pm.05}$	$0.96^{\pm.03}$
SLD-Max	$0.78^{\pm.18}$	$0.80^{\pm.09}$	$0.75^{\pm.17}$	$0.76^{\pm.08}$	$0.79^{\pm.16}$	$0.82^{\pm.08}$	$0.65^{\pm.18}$	$0.63^{\pm.09}$	$0.57^{\pm.17}$	$0.55^{\pm.08}$	$0.84^{\pm.15}$	$0.86^{\pm.07}$	$0.61^{\pm.16}$	$0.64^{\pm.07}$	$0.71^{\pm.19}$	$0.72^{\pm.06}$
SAFREE	$0.76^{\pm.22}$	$0.78^{\pm.21}$	$0.73^{\pm.23}$	$0.74^{\pm.22}$	$0.78^{\pm.21}$	$0.80^{\pm.18}$	$0.63^{\pm.22}$	$0.61^{\pm.21}$	$0.56^{\pm.21}$	$0.54^{\pm.20}$	$0.82^{\pm.18}$	$0.84^{\pm.15}$	$0.60^{\pm.21}$	$0.62^{\pm.20}$	$0.69^{\pm.25}$	$0.70^{\pm.23}$
PurifyGen	$0.75^{\pm.13}$	$0.77^{\pm.19}$	$0.72^{\pm.12}$	$0.74^{\pm.18}$	$0.77^{\pm.11}$	$0.80^{\pm.17}$	$0.64^{\pm.11}$	$0.62^{\pm.17}$	$0.57^{\pm.10}$	$0.55^{\pm.16}$	$0.83^{\pm.10}$	$0.85^{\pm.15}$	$0.61^{\pm.10}$	$0.63^{\pm.16}$	$0.70^{\pm.14}$	$0.71^{\pm.21}$
UCE	$0.79^{\pm.04}$	$0.83^{\pm.06}$	$0.77^{\pm.04}$	$0.81^{\pm.07}$	$0.81^{\pm.04}$	$0.84^{\pm.06}$	$0.68^{\pm.04}$	$0.72^{\pm.06}$	$0.61^{\pm.04}$	$0.65^{\pm.06}$	$0.87^{\pm.03}$	$0.91^{\pm.05}$	$0.65^{\pm.03}$	$0.69^{\pm.05}$	$0.75^{\pm.05}$	$0.80^{\pm.08}$

Appendix

Table 9: Per-category Expected Maximum Inappropriateness (

\downarrow

) on SD v1.5. N: Naive, A: Adversarial.

Method	Hate		Harassment		Violence		Self-harm		Sexual		Shocking		Illegal act.		Overall
Method	N	A	N	A	N	A	N	A	N	A	N	A	N	A	N	A
SD Vanilla	$0.94^{\pm.04}$	$0.97^{\pm.02}$	$0.92^{\pm.07}$	$0.94^{\pm.05}$	$0.90^{\pm.08}$	$0.91^{\pm.06}$	$0.99^{\pm.01}$	$0.97^{\pm.02}$	$0.92^{\pm.06}$	$0.92^{\pm.04}$	$0.99^{\pm.01}$	$0.98^{\pm.01}$	$0.94^{\pm.05}$	$0.93^{\pm.03}$	$0.94^{\pm.01}$	$0.98^{\pm.01}$
SLD-Max	$0.77^{\pm.17}$	$0.79^{\pm.08}$	$0.74^{\pm.16}$	$0.75^{\pm.07}$	$0.78^{\pm.15}$	$0.81^{\pm.07}$	$0.64^{\pm.17}$	$0.62^{\pm.08}$	$0.56^{\pm.16}$	$0.54^{\pm.07}$	$0.83^{\pm.14}$	$0.85^{\pm.06}$	$0.60^{\pm.15}$	$0.63^{\pm.06}$	$0.73^{\pm.24}$	$0.74^{\pm.15}$
SAFREE	$0.75^{\pm.21}$	$0.77^{\pm.20}$	$0.72^{\pm.22}$	$0.73^{\pm.21}$	$0.77^{\pm.20}$	$0.79^{\pm.19}$	$0.62^{\pm.21}$	$0.60^{\pm.20}$	$0.55^{\pm.20}$	$0.53^{\pm.19}$	$0.81^{\pm.19}$	$0.83^{\pm.18}$	$0.59^{\pm.20}$	$0.61^{\pm.19}$	$0.72^{\pm.25}$	$0.75^{\pm.24}$
PurifyGen	$0.76^{\pm.12}$	$0.78^{\pm.18}$	$0.73^{\pm.11}$	$0.75^{\pm.17}$	$0.78^{\pm.10}$	$0.81^{\pm.16}$	$0.65^{\pm.10}$	$0.63^{\pm.16}$	$0.58^{\pm.09}$	$0.56^{\pm.15}$	$0.84^{\pm.09}$	$0.86^{\pm.12}$	$0.62^{\pm.09}$	$0.64^{\pm.15}$	$0.74^{\pm.15}$	$0.73^{\pm.14}$
UCE	$0.78^{\pm.05}$	$0.82^{\pm.07}$	$0.75^{\pm.04}$	$0.79^{\pm.06}$	$0.80^{\pm.03}$	$0.83^{\pm.05}$	$0.67^{\pm.03}$	$0.71^{\pm.05}$	$0.60^{\pm.03}$	$0.64^{\pm.05}$	$0.86^{\pm.03}$	$0.90^{\pm.05}$	$0.64^{\pm.03}$	$0.68^{\pm.05}$	$0.76^{\pm.22}$	$0.77^{\pm.12}$

Appendix

Table 10: Per-category Expected Maximum Inappropriateness (

\downarrow

) on SD v2.0. N: Naive, A: Adversarial.

Method	Hate		Harassment		Violence		Self-harm		Sexual		Shocking		Illegal act.		Overall
Method	N	A	N	A	N	A	N	A	N	A	N	A	N	A	N	A
SD Vanilla	$0.97^{\pm.02}$	$0.99^{\pm.01}$	$0.94^{\pm.03}$	$0.96^{\pm.05}$	$0.92^{\pm.08}$	$0.93^{\pm.06}$	$0.99^{\pm.01}$	$0.99^{\pm.01}$	$0.94^{\pm.05}$	$0.94^{\pm.04}$	$0.98^{\pm.01}$	$1.00^{\pm.00}$	$0.96^{\pm.03}$	$0.95^{\pm.03}$	$0.97^{\pm.02}$	$0.98^{\pm.01}$
SLD-Max	$0.76^{\pm.18}$	$0.79^{\pm.09}$	$0.76^{\pm.16}$	$0.78^{\pm.08}$	$0.80^{\pm.15}$	$0.83^{\pm.07}$	$0.67^{\pm.16}$	$0.65^{\pm.07}$	$0.76^{\pm.14}$	$0.74^{\pm.06}$	$0.86^{\pm.13}$	$0.88^{\pm.06}$	$0.63^{\pm.14}$	$0.66^{\pm.06}$	$0.74^{\pm.02}$	$0.77^{\pm.20}$
SAFREE	$\mathbf{0.74^{\pm.22}}$	$\mathbf{0.77^{\pm.21}}$	$0.74^{\pm.21}$	$0.76^{\pm.20}$	$0.79^{\pm.20}$	$0.81^{\pm.19}$	$0.65^{\pm.20}$	$0.63^{\pm.19}$	$0.74^{\pm.18}$	$0.72^{\pm.17}$	$0.84^{\pm.15}$	$0.86^{\pm.13}$	$0.61^{\pm.18}$	$0.63^{\pm.17}$	$0.75^{\pm.12}$	$0.76^{\pm.14}$
PurifyGen	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–
UCE	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–

Appendix

Table 11: Per-category Expected Maximum Inappropriateness (

\downarrow

) on SD v3.0. N: Naive, A: Adversarial. ‘–’ indicates the method not applicable.

$m$	Inappropriate Probability $\downarrow$	Expected Maximum Inappropriateness $\downarrow$	FID $\downarrow$	Time (s)
1	0.17	0.65	16.20	4.35
2	0.18	0.66	15.95	3.56
3	0.19	0.67	15.76	2.98
4	0.20	0.67	15.81	2.20
5	0.21	0.68	15.76	2.05
6	0.34	0.81	16.90	1.72

Appendix

Table 12: Effect of probe interval

m

on SD v1.5 (Naive set).

$m$	Inappropriate Probability $\downarrow$	Expected Maximum Inappropriateness $\downarrow$	FID $\downarrow$	Time (s)
1	0.15	0.62	14.70	2.05
2	0.16	0.63	14.55	1.68
3	0.17	0.64	14.48	1.42
4	0.23	0.71	16.31	1.15
5	0.31	0.79	18.55	1.05

Appendix

Table 13: Effect of probe interval

m

on SD v3.0 (Naive set).

Figure 3: Cosine similarity between intermediate decoded images $\hat{\boldsymbol{x}}_{0}^{(t)}$ and the final generated image $\boldsymbol{x}_{0}$ across denoising timesteps.

Method	No Attack		$t^{*}=35$		$t^{*}=25$		$t^{*}=15$
Method	IP $\downarrow$	CLIP $\uparrow$	IP $\downarrow$	CLIP $\uparrow$	IP $\downarrow$	CLIP $\uparrow$	IP $\downarrow$	CLIP $\uparrow$
SD Vanilla	0.44	0.31	0.68	0.28	0.72	0.27	0.65	0.29
SLD-Max	0.22	0.30	0.51	0.28	0.58	0.27	0.61	0.28
Ours	0.19	0.31	0.28	0.30	0.33	0.30	0.48	0.29

Appendix

Table 14: Inappropriate Probability (IP

\downarrow

) and CLIP Score (

\uparrow

) under mid-generation latent attack at different attack timesteps

t^{*}

Method	SD v1.5				SD v2.0				SD v3.0
	IP $\downarrow$		EMI $\downarrow$		IP $\downarrow$		EMI $\downarrow$		IP $\downarrow$		EMI $\downarrow$
	Naive	Adv	Naive	Adv	Naive	Adv	Naive	Adv	Naive	Adv	Naive	Adv
SD Vanilla	0.43	0.51	$0.96^{\pm 0.14}$	$0.97^{\pm 0.06}$	0.44	0.49	$0.95^{\pm 0.20}$	$0.99^{\pm 0.04}$	0.45	0.53	$0.97^{\pm 0.05}$	$0.99^{\pm 0.03}$
SLD-Max	0.25	0.22	$0.73^{\pm 0.18}$	$0.74^{\pm 0.07}$	0.24	0.23	$0.75^{\pm 0.22}$	$0.76^{\pm 0.14}$	0.23	0.24	$0.74^{\pm 0.12}$	$0.77^{\pm 0.20}$
SAFREE	0.23	0.22	$0.71^{\pm 0.24}$	$0.72^{\pm 0.22}$	0.22	0.24	$0.73^{\pm 0.28}$	$0.77^{\pm 0.23}$	0.22	0.21	$0.71^{\pm 0.11}$	$0.73^{\pm 0.09}$
PurifyGen	0.24	0.23	$0.72^{\pm 0.13}$	$0.73^{\pm 0.20}$	0.25	0.24	$0.76^{\pm 0.14}$	$0.75^{\pm 0.13}$	–	–	–	–

Appendix

Table 15: Held-out evaluation under Q16 [ 39 ] , a detector never used in the training reward. IP: Inappropriate Probability (

\downarrow

), EMI: Expected Maximum Inappropriateness (

\downarrow

). Results are consistent with Table 1 , confirming that our method does not over-fit to the training detectors.

Category	Hyperparameter	Value
Training	Learning rate	$1\times 10^{-4}$
	Optimizer	Adam
	Mini-batch size	128
	Total episodes	20,000
	Discount factor $\gamma$	0.99
PPO	Clip ratio $\varepsilon$	0.2

Appendix

Table 16: Hyperparameters used in all experiments.

Explore similar work

InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation
Sep 23, 2026Zeyu Wang, Xiaodan Li, Zhiwen Li +2Text-To-Image Guardrail
Alibaba AAIG
Introspective Attention Modulation for Safe Text-to-Image Generation
Jul 16, 2026Basim Azam, Hossein Rahmani, Naveed AkhtarText-To-Image Diffusion Models Text-To-Image
The University of Melbourne, Australia · Lancaster University, United Kingdom
Disciplined Diffusion: Text-to-Image Diffusion Model against NSFW Generation
May 1, 2026Chi Zhang, Changjia Zhu, Xiaowen Li +2Text-To-Image Diffusion Models Safety Filters
Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida, Tampa, FL, USA · Department of Electrical and Computer Engineering, College of Engineering, University of South Florida, Tampa, FL, USA

Sep 23, 2026, cs.CVJ/K move · Enter open · S save

InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation

Zeyu Wang, Xiaodan Li, Zhiwen Li, Yuefeng Chen+1

Alibaba AAIG

Text-To-Image Guardrail Text-To-Video Generation Model Copyright Defenses

Open paper AskPDF arXiv

Modern text-to-image (T2I) models generate high-quality images from arbitrary user prompts, yet they can just as easily produce not-safe-for-work (NSFW) content. Conventional outer guardrails consist of two components: a prompt classifier that checks for risk before generation, and a post-hoc image classifier that checks the fully generated image. In this design, both classifiers operate outside the generation pipeline and do not use the model's own representations. This separation can limit prompt-screening accuracy, while the image-side check runs only after the full generation cost has been spent. Moreover, a flagged prompt can only be rejected, even when it could be adjusted to produce a safe image. In this work, we propose the Inner Guardrail (InGuard), a safety framework that works inside the pipeline on the model's own representations, leaving base-model parameters untouched. First, a risk classifier grades each prompt as unsafe, risky, or benign based on the text encoder's embeddings, with no external language model. Second, SAGE (Soft-gated Asymmetric Guardrail for Embeddings) modifies the embeddings of risky prompts, aiming to return a safe image instead of a refusal. Third, a latent detector checks the one-step clean latent estimate midway through denoising, reaching nearly image-level performance and halting generation when risk is detected. We also construct the RevGen Safety Benchmark to evaluate T2I safety under realistic conditions: 10,000 prompts built through real-image reverse generation, with a rewriting step that supplies controlled intellectual-property (IP) characters, covering graded porn/gore risks, categorical IP risks, and benign negatives. Across five open-weight T2I models, InGuard reaches 97.9-98.8% safety rate, matching or exceeding the outer guardrail, with 57.5-73.5% less benign disturbance, ~3.7x fewer parameters, and 50-55.6% of denoising steps skipped.

Abstract

Explore similar work

InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation

Introspective Attention Modulation for Safe Text-to-Image Generation

Disciplined Diffusion: Text-to-Image Diffusion Model against NSFW Generation