Safety mechanisms for diffusion and flow models have recently been developed along two distinct paths. In robot planning, control barrier functions are employed to guide generative trajectories away from obstacles at every denoising step by explicitly imposing geometric constraints. In parallel, recent data-driven, negative guidance approaches have been shown to suppress harmful content and promote diversity in generated samples. However, they rely on heuristics without clearly stating when safety guidance is actually necessary. In this paper, we first introduce a unified probabilistic framework using a Maximum Mean Discrepancy (MMD) potential for image generation tasks that recasts both Shielded Diffusion and Safe Denoiser as instances of our energy-based negative guidance against unsafe data samples. Furthermore, we leverage control-barrier functions analysis to justify the existence of a critical time window in which negative guidance must be strong; outside of this window, the guidance should decay to zero to ensure safe and high-quality generation. We evaluate our unified framework on several realistic safe generation scenarios, confirming that negative guidance should be applied in the early stages of the denoising process for successful safe generation.
Figures & tables
Figure 1: (a) By incorporating SAFREE ( Yoon et al., 2024 ) and SLD ( Schramowski et al., 2023 ) , our method avoids generating inappropriate images. (b) On artificially memorized SDv2.1 ( Somepalli et al., 2023 ) , it mitigates memorization, with early-stopped negative guidance preserving quality, enhancing diversity, and revealing a critical time window. All images are sampled at the top 5% most similar to the Imagenette training set.
Figure 2: Motivation: 2D flow - matching toy example. (a) A pretrained flow with “negative” data points highlighted in orange. (b) Learned velocity field fθ(x) together with the negative - guidance direction ∇xE(x) . This panel depicts samples at t=0.8 (c) Samples generated with full negative guidance; squared Wasserstein distance to the target distribution (excluding negative regions) W2=1.009 . (d) Samples generated with early - stop negative guidance; squared Wasserstein distance W2=0.937 . Applying full negative guidance either leaves mass near the unsafe set or distorts nearby modes. In contrast, early stopping of the guidance reduces the probability of placing particles near the unsafe region and produces samples that better match the target distribution.
Method
Fine Tuning
Negative Prompt
Negative Guidance
Ring-A-Bell
UnlearnDiff
MMA-Diffusion
COCO
ASR ↓
TR ↓
ASR ↓
TR ↓
ASR ↓
TR ↓
FID ↓
CLIP ↑
SD-v1.4
-
-
-
0.797
0.809
0.809
0.845
0.962
0.956
25.04
31.38
ESD
✓
✗
✗
0.456
0.506
0.422
0.426
0.628
0.640
27.38
30.59
RECE
✓
✗
✗
0.177
0.212
0.284
0.292
0.651
0.664
33.94
30.29
SLD
✗
✓
✗
0.481
0.573
0.629
0.586
0.881
0.882
36.47
29.28
SLD + SafeDenoiser
✗
✓
✓
0.354
0.429
0.526
0.485
0.481
0.549
36.59
29.10
Table 1: Performance comparison on various datasets in safe generation against nudity prompts.
Model
Early Stop
FID ↓
CLIP ↑
AES ↑
Vendi ↑
Recall ↑
Precision ↑
SDv3
-
29.77
31.50
5.554
2.878
0.139
0.883
( λ=1.0 )
SPELL
✗
51.76
28.14
5.190
5.560
0.300
0.530
✓
48.50
28.17
5.051
5.872
0.353
0.521
Ours
✗
36.81
30.47
5.727
3.126
0.119
0.811
✓
31.81
30.78
5.560
3.076
0.135
0.836
Table 2: Performance comparison of ’class-of-image’ task for diversity using ImageNet dataset. ✓indicates negative guidance with early stop =[1.0,0.78] , meanwhile ✗ points out full negative guidance =[1.0,0.0]
Figure 5
Figure 4: Ablation on time windows of negative guidance
Models
Time (s/img)
SD-v1.4
3.18
+ SafeDenoiser ( N=515 )
3.20
+ Ours ( N=515 )
3.22
SAFREE
4.22
+ SafeDenoiser ( N=515 )
4.24
+ Ours ( N=515 )
4.32
Table 4: Wall-clock time.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure D.1: Reference images for safe generation against nudity prompts
Figure F.2: Ablation on time windows of negative guidance for Safe Denoiser
CFG
Model
Budget
Time Windows
FID ↓
CLIP ↑
AES ↑
Recall ↑
Vendi ↑
Converage ↑
Precision ↑
Density ↑
3.5
SDv3
-
-
29.77
31.50
5.554
0.139
2.878
0.578
0.883
1.187
SPELL
0.03
[1.0, 0.78]
32.77
30.68
5.576
0.138
3.105
0.501
0.826
0.991
[1.0, 0.00]
38.23
30.30
5.733
0.115
3.152
0.435
0.794
0.828
1
[1.0, 0.78]
48.50
28.17
5.051
0.353
5.872
0.423
0.521
0.538
[1.0, 0.00]
51.76
28.14
5.190
0.300
5.560
0.370
0.530
0.490
Ours
0.03
[1.0, 0.78]
31.95
30.75
5.564
0.140
3.082
0.520
0.833
1.031
Appendix
Table F.1: Extended performance comparison of ’class-of-image’ task for diversity using ImageNet dataset including CFG =5.0 .
Model
Time Windows
Budget
CLIP ↑
FID ↓
@Sim 95% ↓
Memorized SDv2.1
[1.0, 0.05]
0.03
31.35
43.07
0.317
[1.0, 0.2]
0.03
31.32
40.35
0.324
[1.0, 0.4]
0.03
31.15
36.97
0.334
[1.0, 0.6]
0.03
30.93
35.66
0.328
[1.0, 0.8]
0.03
30.93
32.44
0.338
Appendix
Table F.2: Performance of similarity and image qulaity by varying a time window in memorization experiments.
Figure G.3: Generated images from baselines and our method on nudity prompts. We deliberately chose challenging cases that remain difficult for recent models rather than trivial examples.
Figure G.4: Generated images on challenging nudity prompts under varying negative-guidance windows demonstrate the existence of a critical window. All images are generated by SAFREE + Ours. As the window length increases, nudity attributes unexpectedly reappear. This phenomenon is similar to the third case in Figure 2(c) , where prolonged negative guidance amplifies instability, hindering the denoising process and leaving samples concentrated in unsafe regions. This observation extends to real nudity prompts.
Figure G.5: Generated images on challenging nudity prompts where previous methods fail. All images are generated by SAFREE + Ours. While varying negative-guidance windows shows limited effect for baselines, our method removes nudity while preserving the contextual semantics of the prompts. However, we also observe that larger time windows reduce image fidelity, indicating the importance of a critical window for guidance.
Figure G.6: Style-level intellectual property control for The Scream. our method across different time windows that remove the Munch style while preserving the Barbie concept. Out of time windows, early window maintains image fidelity and effectively avoiding Munch’s style.
Figure G.7: Generated images on artificially memorized SDv2.1 ( Somepalli et al., 2023 ) . All samples are drawn from the top 2% most similar to the Imagenette training set. In each block, the leftmost column shows the generated image, while the subsequent ten columns correspond to the top-1 through top-10 most similar images retrieved from the training split. Baseline models exhibit strong memorization, often reproducing near-duplicates of training images.
Figure G.8: Generated images from our method on artificially memorized SDv2.1 ( Somepalli et al., 2023 ) . As in Figure G.7 , all samples are taken from the top 2% most similar to the Imagenette training set, with the leftmost column showing the generated image and the next ten columns presenting the top-1 to top-10 most similar training images. Unlike baselines, our method mitigates memorization, yielding more diverse generations while still preserving image quality, thanks to early-stopped negative guidance that reveals a critical time window.
Flow-based generative models, such as diffusion models and flow matching models, have achieved remarkable success in learning complex data distributions. However, a critical gap remains for their deployment in safety-critical domains: the lack of formal guarantees that generated samples will satisfy hard constraints. We propose a safety filtering framework that acts as an online shield for any pre-trained generative model. Our key insight is to cooperate with the generative process rather than override it. We define a constricting safety tube that is relaxed at the initial noise distribution and progressively tightens to the target safe set at the final data distribution, mirroring the coarse-to-fine structure of the generative process itself. By characterizing this tube via Control Barrier Functions (CBFs), we synthesize a feedback control input through a convex Quadratic Program (QP) at each sampling step. As the tube is loosest when noise is high and intervention is cheapest in terms of control energy, most constraint enforcement occurs when it least disrupts the model's learned structure. We prove that this mechanism guarantees safe sampling in discrete-time. The minimum-norm control synthesized at each step minimizes the per-step contribution to the KL divergence between the safe and original distributions. Across all experiments, we observe 100% constraint satisfaction. Our framework applies to any pre-trained flow-based sampling scheme requiring no retraining or architectural modifications. We validate the approach across constrained image generation, physically-consistent trajectory sampling, and safe robotic manipulation policies, achieving 100% constraint satisfaction while preserving semantic fidelity.
Darshan Gadginmath, Ahmed Allibhoy, Fabio Pasqualetti
Mitsubishi Electric Research Laboratories Cambridge, Massachusetts · Electrical Engineering and Computer Science University of California, Irvine
Recent work on text diffusion models offers a promising alternative to autoregressive generation, but controlling their safety remains underexplored. Existing safety approaches are geared toward autoregressive models and typically rely on post-hoc filtering or inference-time interventions. These are inadequate for effectively addressing safety risks in text diffusion models. We propose the Safety-Aware Denoiser (SAD), a safety-guidance framework in text diffusion models. The SAD modifies the iterative denoising process such that the text sample at the final denoising step is steered toward provably safe regions of the text space. This inference-time method can integrate safety constraints into the denoiser, avoiding computationally expensive retraining of the underlying diffusion model and enabling flexible, lightweight safety guidance. We evaluate the safety of the generated text using the SAD, with respect to hazard taxonomy, memorization, and jailbreak. Experimental results show that SAD substantially reduces unsafe generations while preserving generation quality, diversity, and fluency, outperforming existing methods. These results demonstrate that our safety guidance during denoising provides an effective and scalable mechanism for enforcing safety in text diffusion models.
Amman Yusuf, Zhejun Jiang, Mijung Park
Department of Computer Science, The University of British Columbia, Vancouver, Canada.
Text-to-image diffusion models remain vulnerable to adversarial prompts that elicit disallowed content, motivating reliable inference-time controls. A popular approach is negative guidance, which subtracts a negative prompt direction with a fixed weight. However, it often forces a safety-fidelity trade-off, causing artifacts or prompt drift when over-applied and failing under attacks when under-applied. Dynamic variants reweight guidance using posterior-odds signals, which can be brittle for open-vocabulary compositional prompts, while lightweight similarity-based methods ignore the evolving image evidence along the denoising trajectory. We introduce Concept Removal Guidance (CRG), a training-free method that estimates unwanted-concept presence at each diffusion step from the model's noise predictions, and adaptively calibrates negative guidance via a closed-form constrained update enforcing a target presence threshold while minimally perturbing the conditional trajectory. Across red-teaming benchmarks, CRG reduces attack success rates while preserving benign fidelity, and extends to additional suppression targets such as artist style and violence without fine-tuning or external classifiers.
Yoonseok Choi, Chaeyoung Oh, Hyunjun Choi +2
Kim Jaechul Graduate School of AI, KAIST, Daejeon, Republic of Korea · Center for Humanoid Research, KIST, Seoul, Republic of Korea