Decoupled Safety Control: A Safety-Control Algorithm for Training-Free Safety Guidance

Huilin Zhou; Ruoxi Cheng; Yuhang Wang; Yuming Liu; Minghao Sun; Ruolong Ma; Lan Zhang

cs.CVOct 15, 2026

Decoupled Safety Control: A Safety-Control Algorithm for Training-Free Safety Guidance

Authors: Huilin Zhou, Ruoxi Cheng, Yuhang Wang, Yuming Liu, Minghao Sun, Ruolong Ma, Lan Zhang

Organizations: University of Science and Technology of China · Fudan University · Tencent · Beijing University of Posts and Telecommunications ·

Abstract

Diffusion models achieve strong performance in text-to-image generation, but they can still produce unsafe content. Training-free safety guidance offers a practical inference-time alternative that intervenes during sampling without modifying model weights. However, many existing safeguards still rely on a coupled safety-control chain. In this chain, the same unsafe condition simultaneously specifies what should be avoided, indicates the current unsafe signal, constructs the intervention, and determines when that intervention remains active. This coupling can work when unsafe factors are explicit, but becomes less reliable once harmful and benign semantics are entangled in the current generation state, often leaving residual unsafe content or unnecessarily degrading benign-generation quality or prompt fidelity. We address this problem with Decoupled Safety Control (DSC), a plug-in safety-control algorithm for training-free safety guidance. DSC reformulates training-free safety guidance as an explicit state-dependent control process, so the executed intervention is determined from the current generation state rather than inherited directly from a single unsafe condition. Experiments on standard safety benchmarks show that DSC improves representative latent- and text-space safeguards, achieving stronger unsafe-content suppression while preserving benign-generation quality and maintaining efficient inference.

Figures & tables

Figure 1: Overview of DSC at each denoising step. DSC organizes runtime evidence extraction, control-state estimation, intervention routing, update construction, and phase-aware execution around the baseline safeguard step.

Method	Weight Update	Training -Free	Decoupled Control	Ring-A-Bell		UnlearnDiff		MMA-Diffusion		COCO
Method	Weight Update	Training -Free	Decoupled Control	ASR $\downarrow$	TR $\downarrow$	ASR $\downarrow$	TR $\downarrow$	ASR $\downarrow$	TR $\downarrow$	FID $\downarrow$	CLIP $\uparrow$
SD-v1.4	-	-	-	0.797	0.809	0.809	0.845	0.962	0.956	25.04	31.38
ESD	$\checkmark$	✗	✗	0.456	0.506	0.422	0.426	0.628	0.640	27.38	30.59
RECE	$\checkmark$	✗	✗	0.177	0.212	0.284	0.292	0.651	0.664	33.94	30.29
SLD	✗	$\checkmark$	✗	0.481	0.573	0.629	0.586	0.881	0.882	36.47	29.28
SLD + SGF	✗	$\checkmark$	✗	0.228	0.294	0.353	0.431	0.297	0.357	36.83	28.13

Table 1: Performance comparison on adversarial nudity benchmarks. Our method improves safety while preserving image quality.

Method	Harassment $\downarrow$	Hate $\downarrow$	Illegal Activity $\downarrow$	Self-harm $\downarrow$	Sexual $\downarrow$	Shocking $\downarrow$	Violence $\downarrow$	Avg. IP $\downarrow$	CLIP $\uparrow$
SD-v1.4	0.269	0.154	0.206	0.319	0.120	0.221	0.274	0.223	29.81
SLD	0.223	0.106	0.161	0.247	0.078	0.158	0.217	0.170	29.65
SLD + Ours	0.148	0.095	0.134	0.156	0.067	0.127	0.190	0.131	29.02
SAFREE	0.182	0.118	0.144	0.183	0.085	0.150	0.206	0.153	28.91
SAFREE + Ours	0.101	0.062	0.062	0.116	0.045	0.085	0.146	0.088	28.38

Table 2: Performance of IP and CLIP on the CoPro dataset. Lower IP and higher CLIP are better.

Method	ASR $\downarrow$	$\Delta$ ASR	TR $\downarrow$	$\Delta$ TR
SAFREE	0.601	$\uparrow$ 0.434	0.618	$\uparrow$ 0.393
w/o runtime evidence extraction	0.394	$\uparrow$ 0.227	0.448	$\uparrow$ 0.223
w/o control-state estimation	0.419	$\uparrow$ 0.252	0.470	$\uparrow$ 0.245
w/o intervention routing	0.452	$\uparrow$ 0.285	0.494	$\uparrow$ 0.269
w/o update construction	0.462	$\uparrow$ 0.295	0.507	$\uparrow$ 0.282
w/o phase-aware execution	0.492	$\uparrow$ 0.325	0.525	$\uparrow$ 0.300

Table 3: Mechanism ablation on MMA-Diffusion.

Figure 2: Runtime control organization on MMA-Diffusion under the same SAFREE pipeline. DSC shifts executed control from the SAFREE-default chain to explicit intervention routing.

Figure 3: Runtime statistics of SAFREE + Ours on MMA-Diffusion.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure C.1: Qualitative comparison on Ring-A-Bell [ Tsai et al., 2024 ] .

Figure C.2: Qualitative comparison on UnlearnDiff [ Zhang et al., 2024 ] .

Method	FID $\downarrow$	CLIP $\uparrow$
SAFREE + Ours	17.14	30.49
SLD + Ours	26.54	29.05

Appendix

Table C.1: Full COCO-30K FID and CLIP results for our method applied to SLD and SAFREE.

Figure C.3: The first 300 generated images from SAFREE + Ours on COCO-30K.

Figure C.4: Qualitative comparison on selected CoPro prompts from the illegal-activity category. Our method guides the outputs toward safer scenes while preserving the broad scene context.

Figure C.5: Qualitative comparison on Ring-A-Bell prompts with SD-v3.

Quantity	SLD	SAFREE
$d_{t}^{-}$	$\epsilon_{t}^{\mathrm{txt}}-\epsilon_{t}^{\mathrm{safe}}$	$\epsilon_{t}^{\mathrm{txt}}-\epsilon_{t}^{\mathrm{filt}}$
$d_{t}^{+}$	$\epsilon_{t}^{\mathrm{txt}}-\epsilon_{t}^{\mathrm{uncond}}$	$\epsilon_{t}^{\mathrm{filt}}-\epsilon_{t}^{\mathrm{uncond}}$

Appendix

Table D.1: Denoising-branch evidence for the two safeguard instantiations.

Parameter	Value
$\tau_{\kappa}$	$0.80$
$\tau_{\eta}$	$0.56$
$\tau_{\omega}$	$0.08$
$[c_{\min},c_{\max}]$	$[0.98,1.28]$ (SLD); $[0.94,1.22]$ (SAFREE)
$c_{\mathrm{ref}}$	$1.0$
$\rho_{\max}$	$0.85$

Appendix

Table D.2: Fixed DSC parameters.

Method	ASR $\downarrow$ [95% CI]	Paired $\Delta$ [95% CI]
SLD + SGF	$0.297\ [0.269,0.326]$	—
SLD + Ours	$\mathbf{0.169}\ [0.147,0.192]$	$-0.128\ [-0.156,-0.101]$
SAFREE + SGF	$0.423\ [0.392,0.454]$	—
SAFREE + Ours	$\mathbf{0.167}\ [0.145,0.190]$	$-0.256\ [-0.288,-0.224]$

Appendix

Table E.1: MMA-Diffusion ASR and prompt-cluster bootstrap 95% confidence intervals. Paired

\Delta

is DSC minus SGF within the same safeguard; a negative value favors DSC.

Evaluation protocol	SLD + SGF	SLD + Ours	SAFREE + SGF	SAFREE + Ours
NudeNet	$0.297$	$\mathbf{0.169}$	$0.423$	$\mathbf{0.167}$
Q16	$0.312$	$\mathbf{0.181}$	$0.437$	$\mathbf{0.174}$
ShieldGemma2	$0.431$	$\mathbf{0.128}$	$0.514$	$\mathbf{0.072}$
Llama Guard 4	$0.402$	$\mathbf{0.121}$	$0.487$	$\mathbf{0.069}$
Human	$0.441$	$\mathbf{0.153}$	$0.479$	$\mathbf{0.131}$

Appendix

Table E.2: MMA-Diffusion unsafe rate

\downarrow

under four automatic evaluation protocols and human annotation. Comparisons are made within each row and safeguard.

Evaluation protocol	SLD	SLD + Ours	SAFREE	SAFREE + Ours
Original classifier (avg. IP)	$0.170$	$\mathbf{0.131}$	$0.153$	$\mathbf{0.088}$
ShieldGemma2	$0.198$	$\mathbf{0.142}$	$0.176$	$\mathbf{0.095}$
Llama Guard 4	$0.189$	$\mathbf{0.138}$	$0.168$	$\mathbf{0.091}$
Human	$0.183$	$\mathbf{0.119}$	$0.130$	$\mathbf{0.107}$

Appendix

Table E.3: CoPro seven-category average under the original IP protocol and additional evaluation protocols (

\downarrow

). Each row compares the safeguard with and without DSC.

Safe T2I Generation

Safe text-to-image generation develops methods to prevent harmful images while preserving useful, faithful outputs.

See all 53 papers

The Safety-Aware Denoiser for Text Diffusion Models
Apr 28, 2026Amman Yusuf, Zhejun Jiang, Mijung ParkDiffusion Model Guidance Diffusion Language Models
Department of Computer Science, The University of British Columbia, Vancouver, Canada.
SafeCtrl: Region-Aware Safety Control for Text-to-Image Diffusion via Detect-Then-Suppress
Apr 5, 2026Lingyun Zhang, Yu Xie, Zhongli Fang +2Safe T2I Generation Adversarial Robustness
School of Computer Science and Technology, Fudan University, China · Institute of Big Data, Fudan University, China · Nanjing University of Information Science and Technology, China
SafeDiffusion-R1: Online Reward Steering for Safe Diffusion Post-Training
May 18, 2026Komal Kumar, Ankan Deria, Abhishek Basu +3Diffusion Model Alignment Diffusion Model Fine-Tuning
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE · Michigan State University (MSU), USA

Apr 28, 2026J/K move · Enter open · S save

The Safety-Aware Denoiser for Text Diffusion Models

Amman Yusuf, Zhejun Jiang, Mijung Park

Department of Computer Science, The University of British Columbia, Vancouver, Canada.

Diffusion Model Guidance Diffusion Language Models

Open paper AskPDF arXiv

Recent work on text diffusion models offers a promising alternative to autoregressive generation, but controlling their safety remains underexplored. Existing safety approaches are geared toward autoregressive models and typically rely on post-hoc filtering or inference-time interventions. These are inadequate for effectively addressing safety risks in text diffusion models. We propose the Safety-Aware Denoiser (SAD), a safety-guidance framework in text diffusion models. The SAD modifies the iterative denoising process such that the text sample at the final denoising step is steered toward provably safe regions of the text space. This inference-time method can integrate safety constraints into the denoiser, avoiding computationally expensive retraining of the underlying diffusion model and enabling flexible, lightweight safety guidance. We evaluate the safety of the generated text using the SAD, with respect to hazard taxonomy, memorization, and jailbreak. Experimental results show that SAD substantially reduces unsafe generations while preserving generation quality, diversity, and fluency, outperforming existing methods. These results demonstrate that our safety guidance during denoising provides an effective and scalable mechanism for enforcing safety in text diffusion models.

Abstract

Safe T2I Generation

Explore similar work

The Safety-Aware Denoiser for Text Diffusion Models

SafeCtrl: Region-Aware Safety Control for Text-to-Image Diffusion via Detect-Then-Suppress

SafeDiffusion-R1: Online Reward Steering for Safe Diffusion Post-Training