SaTPO: Text-to-Image Safety Alignment via Transition Preference Optimization

Ziyi Fang; Jingzhi Li; Xiaojun Jia; Xingxing Wei; Wenqi Ren; Haijun Zhang

cs.CVOct 14, 2026

SaTPO: Text-to-Image Safety Alignment via Transition Preference Optimization

Authors: Ziyi Fang, Jingzhi Li, Xiaojun Jia, Xingxing Wei, Wenqi Ren, Haijun Zhang

Organizations: Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of Science and Technology Beijing · Nanyang Technological University ·

Abstract

Safety alignment of text-to-image diffusion models aims to suppress harmful content without compromising useful generation capabilities. However, preferences over completed images compress multidimensional generation factors into a coarse signal, potentially driving changes beyond those required for safety. To address this, we propose Safety Transition Preference Optimization (SaTPO), which enforces step-wise alignment during denoising. SaTPO leverages the insight that an unsafe prompt and its safe rewrite naturally form a tightly coupled preference pair when conditioned on a shared parent state and noise realization. By anchoring optimization on identical parent states, SaTPO eliminates historical trajectory divergence, confining safety adaptation to an incremental alignment while preserving task-relevant semantics. Multi-scale drift constraints and benign replay further limit behavioral deviation and preserve ordinary generation. We additionally construct CoProPV, a refined dataset ensuring visually verified contrasts. Experiments on Stable Diffusion v1.5 and SDXL demonstrate improved safety-quality trade-offs. Compared with AlignGuard, PerspectiveVision unsafe-detection rates on Inappropriate Image Prompts decrease from 12.70% to 6.00% and from 15.04% to 9.30%, respectively, while FID improves from 32.28 to 25.95 and from 36.50 to 27.51. Warning: this paper includes potentially offensive content.

Figures & tables

Figure 1: Preference construction for diffusion alignment. AlignGuard uses forward-noised completed-image pairs. LPO ranks step-level latent candidates with an external reward model. SaTPO couples one-step transitions from an unsafe prompt and its safe rewrite using a shared parent state and noise. This design enables reward-free, directional preference learning in latent space.

Figure 2: CoProPV label distribution and preference-gap attenuation along denoising trajectories. (a) Actual image safety labels regenerated from CoProV2 prompt pairs on SD-v1.5 and SDXL. (b) Evolution of the mean SafetyLRM score difference (

\text{preferred}-\text{rejected}

) across denoising timesteps for SDXL child latents. As denoising progresses (

t\to 0

), the preference gap diminishes, providing empirical support for SaTPO’s progressive trajectory regularization.

Figure 3: Overview of SaTPO. SaTPO converts prompt-level safety counterfactuals into coupled transition preferences under shared states and noise, enabling reward-free directional safety alignment. Multi-scale drift regularization constrains deviations at both stepwise and trajectory levels, while benign replay preserves the model’s original generation capability during safety alignment.

Figure 4: Selected qualitative comparisons across seven risk categories on SD-v1.5 and SDXL. Methods share prompts and random seeds within each backbone. The examples illustrate reduced harmful content with retention of selected benign attributes.

	Method	I2P-5000				UD-934				NSFW-Sexual-1k
	Method	PV	Q16	NudeNet	IP	PV	Q16	NudeNet	IP	PV	Q16	NudeNet	IP
SD-v1.5	Base	39.28	36.54	2.70	38.24	40.58	47.32	4.50	49.68	88.70	23.80	20.80	37.30
	SLD-Medium	23.02	16.96	1.44	18.04	23.66	20.45	2.25	22.48	75.80	6.80	15.30	19.90
	SLD-Max	9.78	5.36	0.62	5.96	10.39	8.57	1.07	9.31	42.40	1.00	4.80	5.10
	SAFREE	–	–	–	–	–	–	–	–	50.20	19.30	7.50	22.80
	ESD-u	35.96	33.48	2.96	35.22	37.26	40.69	2.89	41.65	83.90	25.50	15.50	34.60

Table 1: Safety comparison on three prompt sets. All entries are detection rates (%,

\downarrow

). Bold and underline denote the best and second-best distinct result within each backbone and metric, including ties. Dashes indicate unevaluated settings. The text specifies target concepts and backbone coverage.

Figure 5: Benign generation across two backbones. (a) Examples share the displayed prompt across Base, AlignGuard, and SaTPO. (b) Mean CLIP text–image similarity ( $\uparrow$ ) and FID ( $\downarrow$ ) on COCO-5000, with real COCO images as the FID reference. Base gives the unaligned reference.

Table 7

Table 4: Effects of benign replay and timestep weighting on safety and benign generation.

Table 5: Effects of the parent-state reset interval and parent-state sharing on preference learning.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	SD-v1.5	SDXL
Training resolution	$512\times 512$	$1024\times 1024$
Collected denoising steps $\|\mathcal{T}\|$	20	20
DDIM stochasticity $\eta$ / CFG scale $g$	$1.0\;/\;7.5$	$1.0\;/\;7.5$
Sampling batch / pair minibatch	$8\;/\;8$	$8\;/\;4$
LoRA rank	8	64
Learning rate	$10^{-5}$	$10^{-5}$

Appendix

Table 6: SaTPO training settings for the evaluated checkpoints. Penalty parameters refer to Equations ( 6 )–( 11 ).

Method	Single GPU	Training time (GPU-hours)
		SD-v1.5	SDXL
AlignGuard	RTX 5880 Ada	$\approx 14$	$\approx 126$
SafetyBPO	RTX 4090	$\approx 1.5$	Not reported
SaTPO	A800 80GB	10.11	$\approx 76.95$

Appendix

Table 7: Training time under each method’s reported hardware and configuration. All entries use one GPU. AlignGuard and SafetyBPO values come from their original papers; SaTPO values come from our training records. GPU-hours do not normalize hardware performance.

Method (IP)	Harass.	Hate	Illegal	Self-harm	Sexual	Shocking	Violence	Overall
Base	31.23	38.87	35.33	40.60	23.74	54.46	43.35	38.24
SLD-Medium	16.42	19.84	14.05	15.56	6.74	32.81	21.48	18.04
SLD-Max	6.01	5.26	4.17	4.87	1.50	11.50	7.80	5.96
ESD-u	27.86	36.03	32.82	37.08	24.28	48.10	40.15	35.22
UCE	27.27	34.82	26.56	31.12	18.18	47.54	34.65	31.20
RECE	17.30	22.67	15.02	13.67	11.12	23.55	18.67	16.88

Appendix

Table 8: SD-v1.5 category-wise detection rates on I2P-5000 (%,

\downarrow

). Bold and underline rank methods within each metric and category. Category sizes, in column order, are 682, 247, 719, 739, 935, 896, and 782. Overall is sample-weighted over 5,000 images.

Method (IP)	Harass.	Hate	Illegal	Self-harm	Sexual	Shocking	Violence	Overall
Base	29.18	37.65	33.52	40.60	16.58	47.10	34.78	33.64
ESD-u	27.86	40.08	33.66	40.19	15.40	45.65	35.04	33.10
AlignGuard	8.21	6.07	10.57	12.04	3.85	14.06	9.34	9.42
LPO	26.98	41.70	28.09	37.62	14.33	43.30	31.84	30.76
SaTPO	3.96	5.67	4.45	5.82	2.35	8.26	5.63	5.12
Method (PV)	Harass.	Hate	Illegal	Self-harm	Sexual	Shocking	Violence	Overall

Appendix

Table 9: SDXL category-wise detection rates on I2P-5000 (%,

\downarrow

). Bold and underline rank methods within each metric and category. Category sizes, in column order, are 682, 247, 719, 739, 935, 896, and 782. Overall is sample-weighted over 5,000 images.

Exposed region	Base	SLD-Medium	SLD-Max	SAFREE	ESD-u	UCE	RECE	AlignGuard	SafetyBPO	SaTPO
Buttocks	234	199	102	66	163	108	6	29	7	16
Female breast	400	283	70	77	281	293	12	30	15	14
Female genitalia	74	46	11	17	52	41	0	6	0	1
Male breast	85	82	31	54	80	46	0	6	0	1
Anus	6	0	0	2	4	0	0	0	0	0
Feet	185	139	88	185	187	125	26	151	42	94

Appendix

Table 10: SD-v1.5 NudeNet exposure counts on 1,000 NSFW-Sexual-1k images. Each class is counted once per image; classes may overlap. Bold and underline indicate the lowest and next distinct counts, not a ranking of safety for non-genital regions.

Exposed region	Base	SAFREE	ESD-u	AlignGuard	LPO	SaTPO
Buttocks	102	61	98	58	115	21
Female breast	178	41	165	64	188	26
Female genitalia	26	5	21	8	16	2
Male breast	36	12	35	4	21	2
Anus	0	0	0	0	0	0
Feet	184	100	188	173	164	155

Appendix

Table 11: SDXL NudeNet exposure counts on 1,000 NSFW-Sexual-1k images. Each class is counted once per image; classes may overlap. Bold and underline indicate the lowest and next distinct counts, not a ranking of safety for non-genital regions.

	Method	MMA-Diffusion					Ring-A-Bell					SneakyPrompt
	Method	ASR	PV	Q16	NN	IP	ASR	PV	Q16	NN	IP	ASR	PV	Q16	NN	IP
SD-v1.5	Base	93.90	92.90	19.90	36.20	48.70	93.67	88.61	15.19	17.72	32.91	76.50	73.00	27.50	11.00	34.00
	AlignGuard	22.50	20.70	0.90	1.80	2.70	41.77	37.97	3.80	5.06	8.86	20.50	19.50	1.50	0.50	2.00
	SafetyBPO	15.70	14.00	0.60	3.80	4.00	7.59	2.53	2.53	2.53	5.06	9.00	9.00	0.00	1.00	1.00
	SaTPO	6.80	6.20	0.90	0.30	1.20	1.27	0.00	0.00	1.27	1.27	3.00	3.00	0.00	0.00	0.00
SDXL	Base	68.50	65.80	18.50	2.90	20.70	79.75	75.95	8.86	13.92	21.52	64.00	60.00	20.00	1.50	21.50

Appendix

Table 12: Fixed-prompt adversarial evaluation (%,

\downarrow

). ASR is

\mathrm{PV}\cup\mathrm{IP}

; NN denotes NudeNet. Bold and underline rank methods within each backbone, attack, and metric. MMA-Diffusion, Ring-A-Bell, and SneakyPrompt use 1,000, 79, and 200 prompts, respectively.

Table 17

Parent $t$	Safe/Safe		Safe/Unsafe		Unsafe/Unsafe		Unsafe/Safe
Parent $t$	Pref.	Gap	Pref.	Gap	Pref.	Gap	Pref.	Gap
951	85.71	+2.3553	92.14	+3.2182	87.86	+2.7746	87.14	+2.1221
851	86.43	+1.5343	87.86	+2.4755	85.00	+1.6670	78.57	+1.4626
751	85.00	+0.7654	86.43	+1.2624	90.71	+0.8521	77.86	+0.6379
651	82.86	+0.3561	88.57	+0.7382	80.00	+0.4452	75.00	+0.2521
601	77.86	+0.2598	83.57	+0.5237	79.29	+0.3121	75.71	+0.2282

Appendix

Table 16: SafetyLRM diagnostic (140 pairs per group). Pref. is agreement with prompt-source ordering (%, including ties); Gap is the mean score for the safe-rewrite candidate minus that for the unsafe-prompt candidate. Groups follow the corresponding complete-image PV labels.

Figure 7: Additional qualitative comparisons on SD-v1.5 (top) and SDXL (bottom). Rows compare the indicated base and safety-aligned models. Columns show corresponding examples across methods.

Figure 8: Additional qualitative comparisons for sexual-content prompts on SD-v1.5 (top) and SDXL (bottom). Rows compare the indicated models on corresponding examples. Black NSFW overlays mask explicit regions.

Preference Optimization for Diffusion Models

See all 17 papers

Introspective Attention Modulation for Safe Text-to-Image Generation
Jul 16, 2026Basim Azam, Hossein Rahmani, Naveed AkhtarDiffusion Transformer T2I Generation
The University of Melbourne, Australia · Lancaster University, United Kingdom
GuardPaint:SpeculativeSafetyDecodingforText-to-ImageGeneration
Aug 22, 2026Shreyash Dhoot, Paras Dhiman, Arsh Abbas Naqvi +4Safe T2I Generation Adversarial Attacks on Diffusion Models
Pragya Lab, BITS Pilani Goa, India · Apple, USA · Meta, USA
SafeCtrl: Region-Aware Safety Control for Text-to-Image Diffusion via Detect-Then-Suppress
Apr 5, 2026Lingyun Zhang, Yu Xie, Zhongli Fang +2Safe T2I Generation Adversarial Robustness
School of Computer Science and Technology, Fudan University, China · Institute of Big Data, Fudan University, China · Nanjing University of Information Science and Technology, China

Jul 16, 2026J/K move · Enter open · S save

Introspective Attention Modulation for Safe Text-to-Image Generation

Basim Azam, Hossein Rahmani, Naveed Akhtar

The University of Melbourne, Australia · Lancaster University, United Kingdom

Diffusion Transformer T2I Generation Safe T2I Generation Attention Control in Diffusion Models Controllable Image Generation T2I Diffusion Models+1

Open paper AskPDF arXiv

State-of-the-art flow based text-to-image (T2I) models exhibit remarkable generative abilities but remain vulnerable to producing unsafe content. Prior safety efforts range from concept erasure and prompt filtering to classifier-based gating. However, simple techniques like parameter efficient adaptations of the models easily bypass such guardrails. We introduce a unique principled approach that achieves safety by regulating the model's attention dynamics through inference-time introspection, exhibiting intrinsic robustness. Our method analyzes and rebalances attention activations throughout image synthesis, steering generations away from unsafe concepts while preserving semantic alignment. This introspective control ensures safety of deployed models. Across standard and adversarial safety benchmarks, our approach achieves remarkable safety scores while maintaining or even improving alignment and perceptual quality. Our results reveal that attention-space regulation offers a considerably more promising path to safer diffusion transformer based image generation than the existing concept erasing mechanism.Our code can be accessed at https://basim-azam.github.io/iam/

Abstract

Preference Optimization for Diffusion Models

Explore similar work

Introspective Attention Modulation for Safe Text-to-Image Generation

GuardPaint:SpeculativeSafetyDecodingforText-to-ImageGeneration

SafeCtrl: Region-Aware Safety Control for Text-to-Image Diffusion via Detect-Then-Suppress