Text-to-image diffusion models have rapidly transitioned from research prototypes to widely deployed tools. Along with their impressive capabilities, these models can also generate harmful or undesirable content (e.g., NSFW imagery), motivating practical mechanisms for removing such concepts. Negative prompting, which uses the negative-prompt prediction as the guidance anchor, is among the most widely used mechanisms for steering sampling trajectories away from unwanted concepts. However, for jailbreak prompts, it can fail in a regime we refer to as guidance collapse, where the model's denoising predictions for the prompt and negative prompt become similar, so that the guidance difference driving negative prompting vanishes and the guided prediction degenerates to the negative-prompt prediction, undermining suppression. To fix this issue, we propose TraSCE (Trajectory Steering for ConcEpt Suppression), a training-free inference-time method that makes negative prompting reliable under guidance collapse. TraSCE combines two complementary components: (i) unconditional anchoring, which makes the fallback under guidance collapse a neutral unconditional prediction rather than the negative-prompt prediction, and (ii) an alignment-localized steering step, which deliberately induces this collapse for adversarial prompts and steers the sample to the unconditional fallback, while remaining minimally intrusive on benign prompts. We evaluate TraSCE on adversarial prompt benchmarks targeting nudity and violence, and further demonstrate its applicability to suppressing artistic styles and objects. Across these settings, TraSCE substantially improves the robustness of negative prompting to jailbreak prompts without any training, weight updates, or pre-collected concept-specific data (prompts or images), while keeping the impact on general image quality small.
Figures & tables
Figure 2: Empirical evidence of guidance collapse under jailbreak prompts. (Left) Per-timestep alignment gap dt=∥ϵ^p−ϵ^np∥ for benign (blue) versus adversarial (red) prompts; adversarial prompts yield consistently smaller gaps. (Right) Probability P(dt<δ) that the alignment gap falls below a threshold; adversarial prompts fall below every threshold far more often than benign prompts.
Method
Ring-A-Bell
MMA-Diffusion ↓
P4D ↓
UnLearnDiffAtk ↓
I2P ↓
COCO
K77 ↓
K38 ↓
K16 ↓
FID ↓
CLIP ↑
SDv1.4
85.26
87.37
93.68
95.7
98.7
69.7
17.8
16.71
0.304
SA ( Heng & Soh, 2024 )
63.15
56.84
56.84
9.30
47.68
12.68
2.81
25.80
0.297
CA ( Kumari et al., 2023 )
86.32
91.58
89.47
9.90
10.60
5.63
1.04
24.12
0.301
ESD ( Gandikota et al., 2023 )
20.00
29.47
35.79
12.70
9.27
15.49
2.87
18.18
0.302
MACE ( Lu et al., 2024 )
2.10
0.0
0.0
0.50
2.72
2.82
1.51
16.80
0.287
Table 1: Comparison with baseline defenses under adversarial prompts targeting nudity generation: Ring-A-Bell ( Tsai et al., 2023 ) , MMA-Diffusion ( Yang et al., 2024 ) , P4D ( Chin et al., 2024 ) , UnLearnDiffAtk ( Zhang et al., 2024c ) , and I2P ( Schramowski et al., 2023 ) . We report the attack success rate (ASR; lower is better). Nudity detection is performed using NudeNet ( Bedapudi, 2019 ) with a threshold of 0.45 , using the same detector and threshold for all methods. See Appendix B for detector versions and benchmark-specific settings. Rows are grouped by deployment constraint: gray = training-based fine-tuning (requires training and weight updates); blue = training-free weight-editing (updates weights without training); green = inference-time (no training and no weight updates).
Figure 3: Qualitative comparisons under jailbreak prompts targeting nudity generation. TraSCE reduces nudity content while better preserving prompt semantics and visual quality. Additional examples are provided in Appendix Figure 12 .
Method
ASR ↓
SDv1.4
99.6
FNM ( Zhang et al., 2024a )
98.8
CA ( Kumari et al., 2023 )
100
ESD ( Gandikota et al., 2023 )
86.0
UCE ( Gandikota et al., 2024 )
89.8
RECE ( Gong et al., 2024 )
89.2
Table 2: Results on suppressing violence on the adversarial Ring-A-Bell-Union ( Tsai et al., 2023 ) prompt set. We report attack success rate (ASR; lower is better) measured by the Q16 detector ( Schramowski et al., 2022 ) , using the same detector and settings for all methods (see Appendix B for details). COCO-30K FID/CLIP for all methods are reported in Table 1 . Note that SLD-Max attains lower ASR here but with substantially degraded general image quality.
Figure 4: Comparison under adversarial prompts targeting Van Gogh style found by Ring-A-Bell ( Tsai et al., 2023 ) . TraSCE reduces Van Gogh-style artifacts while preserving the prompt content.
Method
Remove “Van Gogh”
Remove “Kelly McKernan”
ACCe↓
ACCu↑
ACCe↓
ACCu↑
SDv1.4
100.0
94.93
70.0
67.08
CA ( Kumari et al., 2023 )
85.00
96.00
55.00
69.23
ESD-x ( Gandikota et al., 2023 )
100.0
89.18
81.25
69.33
RECE ( Gong et al., 2024 )
75.00
92.40
30.00
68.75
UCE ( Gandikota et al., 2024 )
80.00
91.13
55.00
63.75
Table 3: Results on artistic style suppression while maintaining unrelated styles. ACCe is the rate at which the erased style is still recognized as the target style (lower is better), and ACCu is the recognition accuracy for unrelated styles (higher is better). Style classification is performed by GPT-4o following ( Yoon et al., 2024 ) .
Figure 5: Qualitative results on suppressing the object “car” under an adversarial prompt generated by Ring-A-Bell ( Tsai et al., 2023 ) . TraSCE suppresses the target object while preserving overall image quality.
Figure 6: Trajectory analysis of TraSCE (object erasure example). Prompt: “photo of a guitar”; negative concept: “guitar”. (a) Distance to the negative-concept direction, ∥ϵ^−ϵ^np∥22 , across timesteps, where ϵ^ denotes the guided noise prediction used by each method. The unconditional-anchored formulation (Eq. ( 3 )) keeps the guided direction farther from the negative-concept direction than standard negative prompting (Eq. ( 2 )) and SLD-Max, especially in later steps. (b) Loss values Lt (Eq. ( 4 )) with/without localized guidance, showing that the loss-based step reduces the discrepancy between ϵθ(xt,t,ep) and ϵθ(xt,t,enp) .
Negative prompting
Steering
Ring-A-Bell (ASR)
FID ↓
Std. NP
UA-NP
step
K77 ↓
K38 ↓
K16 ↓
✓
17.89
28.42
34.74
18.33
✓
✓
10.63
10.63
13.82
18.45
✓
4.21
10.52
11.57
18.59
✓
✓
1.05
2.10
2.10
17.41
Table 4: Component ablation of TraSCE on Ring-A-Bell (nudity) and COCO-30K. We compare standard negative prompting (Std. NP; Eq. ( 2 )) and unconditional-anchored negative prompting (UA-NP; Eq. ( 3 )), with/without alignment-localized steering. Combining unconditional anchoring with localized guidance reduces ASR while keeping FID close to the base model.
Method
Ring-A-Bell
MMA-Diffusion ↓
P4D ↓
UnLearnDiffAtk ↓
COCO
K77 ↓
K38 ↓
K16 ↓
FID ↓
CLIP ↑
FLUX.1-dev
82.11
82.11
83.16
11.80
49.01
23.94
20.41
0.309
TraSCE (ours)
7.37
3.16
9.47
0.80
2.65
2.11
21.41
0.309
Table 5: Evaluation on FLUX.1-dev under adversarial prompt benchmarks. Ring-A-Bell and MMA-Diffusion are black-box prompt sets, while P4D and UnLearnDiffAtk are static prompt sets produced by white-box attacks. We report ASR (lower is better) and COCO-30K FID/CLIP to assess the trade-off between safety and general image quality.
Method
Detected Nudity (Quantity)
COCO
Common
Female
Male
Total ↓
FID ↓
CLIP ↑
FLUX.1-dev
406
161
38
605
20.41
30.87
CA (Model-based) ( Kumari et al., 2023 )
253
65
26
344
22.66
29.05
CA (Noise-based) ( Kumari et al., 2023 )
290
72
28
390
23.07
28.73
ESD ( Gandikota et al., 2023 )
329
145
32
506
23.08
28.44
UCE ( Gandikota et al., 2024 )
122
39
12
173
30.71
24.56
Table 6: Assessment of nudity suppression on FLUX.1-dev. (Left) Number of images flagged as explicit by NudeNet on I2P ( Schramowski et al., 2023 ) (lower is better). (Right) FID and CLIP score on COCO-30K ( Lin et al., 2014 ) . Following Gao et al. (2025) , we use a NudeNet threshold of 0.6.
Method
Inference time (s)
SDv1.4 w/ Std. NP
5.45
SLD
6.81
SAFREE
6.90
TraSCE (UA-NP; no steering)
6.80
TraSCE (UA-NP + steering)
14.29
Table 7: Comparing inference time (A100 GPU).
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Ring-A-Bell (ASR)
FID ↓
K77 ↓
K38 ↓
K16 ↓
Classifier guidance
w/ CLIP
13.82
23.40
24.46
19.82
w/ NudeNet
7.44
10.63
19.14
51.67
Alignment-localized steering
10.63
10.63
13.82
18.45
Appendix
Table 8: Ablation on guidance strategies for reducing nudity generations on Ring-A-Bell Tsai et al. (2023) . To isolate the effect of the guidance signal, all methods use the same negative prompting formulation (Eq. 2 ) and differ only in the additional guidance term. We report ASR (lower is better) and COCO-30K FID (lower is better). NudeNet-based classifier guidance achieves lower ASR but severely degrades image quality, whereas our alignment-localized steering improves robustness with much smaller FID degradation.
Figure 7: Visual comparison between directly minimizing an MSE loss and using our localized exponential loss (Eq. ( 4 )). The exponential loss better preserves perceptual quality by reducing unnecessary interference when the two predictions are far apart.
Method
Ring-A-Bell (ASR)
FID ↓
CLIP ↑
K16 ↓
K38 ↓
K77 ↓
Hard gate ( τ=0.5 )
17.80
15.79
8.42
19.64
0.298
Hard gate ( τ=1.0 )
9.47
15.79
7.37
18.54
0.298
Hard gate ( τ=1.5 )
5.26
8.42
6.32
16.51
0.296
Hard gate ( τ=3.0 )
0.00
0.00
1.05
25.18
0.265
Soft gate (ours)
1.05
2.10
2.10
17.41
0.299
Appendix
Table 9: Comparison between hard-threshold gating (sweeping τ ) and our Gaussian soft gate on Ring-A-Bell (nudity) and COCO-30K. Lower ASR and lower FID are better; higher CLIP is better.
Method
Ring-A-Bell (ASR)
FID ↓
CLIP ↑
K16 ↓
K38 ↓
K77 ↓
Std. NP ( s=7.5 )
17.89
28.42
34.74
18.33
0.301
Std. NP ( s=20 )
57.89
49.47
42.11
25.33
0.299
TraSCE (ours)
1.05
2.10
2.10
17.41
0.299
Appendix
Table 10: Comparison against simply increasing the classifier-free guidance (CFG) scale on Ring-A-Bell (nudity) and COCO-30K. Increasing the CFG scale does not defend against jailbreak prompts. The repulsion from the negative-prompt prediction has magnitude sδ , so it vanishes as the alignment gap δ→0 under jailbreak prompts, no matter how large the scale s is (Proposition 1 ). Lower ASR and FID are better; higher CLIP is better.
Class name
Accuracy of erased class (%) ↓
Accuracy of other classes (%) ↑
SD
ESD-u
UCE
RECE
SD-NP
TraSCE (ours)
SD
ESD-u
UCE
RECE
SD-NP
TraSCE (ours)
Cassette Player
15.6
0.6
0.0
0.0
4.6
0.0
85.1
64.5
90.3
90.3
64.1
62.8
Chain Saw
66.0
6.0
0.0
0.0
25.2
0.2
79.6
68.2
76.1
76.1
50.9
49.7
Church
73.8
54.2
8.4
2.0
21.2
0.2
78.7
71.6
80.2
80.5
58.4
57.5
English Springer
92.5
6.2
0.2
0.0
0.0
0.0
76.6
62.6
78.9
77.8
63.6
62.8
French Horn
99.6
0.4
0.0
0.0
0.0
0.0
75.8
49.4
77.0
77.0
58.0
54.9
Appendix
Table 11: Results on suppressin objects from Imagenette Howard (2019) , evaluated using a pretrained ResNet-50 classifier. Prompts follow the standard template “Image of a {Object}”. We report (i) accuracy on the erased class (lower is better) and (ii) average accuracy on the remaining classes (higher is better). Note that ESD, UCE, and RECE update model weights, while TraSCE is training-free and does not modify weights.
Figure 8: Hyperparameter sensitivity analysis.
Figure 9: Minimal-invasiveness study using the same examples as Fig. 1 in Amara et al. (2025) . Rows 1 & 3 use prompts containing the suppressed concept (“koala”, “Sun”); Rows 2 & 4 use benign prompts describing closely related concepts (a sloth for “koala”, the moon for “Sun”). TraSCE preserves the benign concept while still suppressing the target, indicating higher specificity.
Figure 10: Concept Suppression with semantically related negative prompts. We suppress a target concept using a negative prompt that is semantically related but not an exact match (“swimsuit” vs. “bikini”) on SDv1.4. TraSCE reduces the target concept even when the prompt and the negative prompt use different but closely related terms (e.g., broader/narrower terms).
Figure 11: Multi-concept suppression on FLUX.1-dev. Prompt: “A depiction of a starry night over a quiet town, reminiscent of Van Gogh’s famous painting”.
Category
TraSCE
ESD
SLD
Least Resembles Van Gogh Style
45.56%/62.5%
17.93%/0.0%
36.41%/37.5%
Best Image Quality
50.54%/100.0%
33.15%/0.0%
16.30%/0.0%
Appendix
Table 12: Human evaluation of artistic style suppression (23 participants; 8 prompts). For each prompt, participants compare outputs from TraSCE, ESD ( Gandikota et al., 2023 ) , and SLD ( Schramowski et al., 2023 ) and vote for (i) the image that least resembles Van Gogh style while preserving prompt semantics, and (ii) the best image quality. We report total vote share across all prompts / percentage of prompts where the method received the most votes .
Figure 12: Additional qualitative comparisons under adversarial prompts aimed at generating nudity content. Many jailbreak prompts are semantically ill-formed (e.g., dominated by random token sequences), for which the intended output is not well-defined; in such cases, TraSCE prioritizes suppression and may produce less semantically specific or less coherent images.
Figure 13: Qualitative results on suppressing the artistic style of Kelly McKernan for the prompt “Whimsical fairy tale scene by Kelly McKernan”, while preserving unrelated styles (Thomas Kinkade, Kilian Eng, and Ajin: Demi-Human). TraSCE better preserves unrelated styles and prompt semantics while suppressing the target style.
Figure 14: Impact on general image generation quality on COCO-30K prompts. TraSCE preserves overall image quality and prompt alignment on non-adversarial prompts.
Text-to-image diffusion models remain vulnerable to adversarial prompts that elicit disallowed content, motivating reliable inference-time controls. A popular approach is negative guidance, which subtracts a negative prompt direction with a fixed weight. However, it often forces a safety-fidelity trade-off, causing artifacts or prompt drift when over-applied and failing under attacks when under-applied. Dynamic variants reweight guidance using posterior-odds signals, which can be brittle for open-vocabulary compositional prompts, while lightweight similarity-based methods ignore the evolving image evidence along the denoising trajectory. We introduce Concept Removal Guidance (CRG), a training-free method that estimates unwanted-concept presence at each diffusion step from the model's noise predictions, and adaptively calibrates negative guidance via a closed-form constrained update enforcing a target presence threshold while minimally perturbing the conditional trajectory. Across red-teaming benchmarks, CRG reduces attack success rates while preserving benign fidelity, and extends to additional suppression targets such as artist style and violence without fine-tuning or external classifiers.
Yoonseok Choi, Chaeyoung Oh, Hyunjun Choi +2
Kim Jaechul Graduate School of AI, KAIST, Daejeon, Republic of Korea · Center for Humanoid Research, KIST, Seoul, Republic of Korea
Text-to-image (T2I) models have become increasingly capable of generating high-quality images. Yet, enforcing the explicit absence of a specified object or attribute remains a fundamentally challenging problem. Existing approaches, including prompt negation, post-hoc editing, and negative guidance, remain insufficient for explicit concept suppression, often failing to remove the target concept or degrading overall image quality. To this end, we propose Orthogonal Negative Guidance in attention feature space, a training-free method that operates in the attention output space of MM-DiT-based T2I transformers. Our method orthogonalizes negative-prompt attention features with respect to positive-prompt features and subtracts only the orthogonal component, suppressing unwanted concepts while preserving desired semantics. Experiments on FLUX-dev and FLUX-schnell show that our method achieves favorable trade-offs between concept suppression, prompt alignment, and image quality. In human evaluation, our method outperforms the second-best baseline by 18.78%. We further show that our method supports multi-concept suppression and adjustable concept suppression.
Jungmin Ko, Jungwon Park, Jimyeong Kim +3
Interdisciplinary Program in Artificial Intelligence, Seoul National University · Research Institute for Convergence Science, Seoul National University · Daegu Gyeongbuk Institute of Science and Technology +3
As text-to-image diffusion models grow increasingly prevalent, the ability to remove specific concepts-mostly explicit content and many copyrighted characters or styles-has become essential for safety and compliance. Existing unlearning approaches often require costly re-training, modify parameters at the cost of degradation of unrelated concept fidelity, or depend on indirect inference-time adjustment that compromise the effectiveness of concept erasure. Inspired by the success of energy-guided sampling for preservation of the condition of diffusion models, we introduce Energy-Guided Latent Optimization for Concept Erasure (EGLOCE), a training-free approach that removes unwanted concepts by re-directing noisy latent during inference. Our method employs a dual-objective framework: a repulsion energy that steers generation away from target concepts via gradient descent in latent space, and a retention energy that preserves semantic alignment to the original prompt. Combined with previous approaches that either require erroneous modified model weights or provide weak inference-time guidance, EGLOCE operates entirely at inference and enhances erasure performance, enabling plug-and-play integration. Extensive experiments demonstrate that EGLOCE improves concept removal while maintaining image quality and prompt alignment across baselines, even with adversarial attacks. To the best of our knowledge, our work is the first to establish a new paradigm for safe and controllable image generation through dual energy-based guidance during sampling.