cs.CVMar 9, 2026
SavePrototype-Guided Concept Erasure in Diffusion Models
Organizations: Fudan University · National University of Singapore
Abstract
Concept erasure is extensively utilized in image generation to prevent text-to-image models from generating undesired content. Existing methods can effectively erase narrow concepts that are specific and concrete, such as distinct intellectual properties (e.g. Pikachu) or recognizable characters (e.g. Elon Musk). However, their performance degrades on broad concepts such as sexual'' or violent'', whose wide scope and multi-faceted nature make them difficult to erase reliably. To overcome this limitation, we exploit the model's intrinsic embedding geometry to identify latent embeddings that encode a given concept. By clustering these embeddings, we derive a set of concept prototypes that summarize the model's internal representations of the concept, and employ them as negative conditioning signals during inference to achieve precise and reliable erasure. Extensive experiments across multiple benchmarks show that our approach achieves substantially more reliable removal of broad concepts while preserving overall image quality, marking a step towards safer and more controllable image generation.
Figures & tables
Figure 2: Broad concepts such as violence encompass multiple semantic modes, including bloodshed (first row), gunfights (second row), and riots (third row). Existing methods such as Safree [ 48 ] and TRCE [ 4 ] erase only part of this spectrum, resulting in incomplete erasure. We highlight this unreliability as a core limitation of prior approaches and introduce our method, which achieves more comprehensive and reliable removal by explicitly modeling the full breadth of a target concept.
| Method | Hate | Harassment | Illegal Activity | Self-harm | Sexual | Shocking | Violence | Overall |
|---|---|---|---|---|---|---|---|---|
| SD v1.4 [ 36 ] | 21.2% | 19.7% | 19.4% | 35.5% | 54.5% | 42.1% | 40.1% | 35.6% |
| ESD* [ 9 ] | 3.5% | 6.4% | 16.7% | 11.1% | 16.4% | 16.1% | 6.3% | 12.2% |
| RECE* [ 11 ] | 4.3% | 6.1% | 6.1% | 8.5% | 8.6% | 9.7% | 14.2% | 8.5% |
| TRCE [ 4 ] | 7.8% | 6.2% | 3.4% | 5.0% | 1.7% | 8.2% | 6.2% | 5.7% |
| SLD* [ 39 ] | 41.1% | 20.1% | 19.4% | 19.2% | 22.9% | 16.0% | 19.7% | 35.6% |
| Safree [ 48 ] | 4.8% | 9.8% | 10.9% | 7.2% | 5.3% | 13.7% | 9.6% | 8.8% |
Table 1: Performance comparison of baselines on the I2P dataset [ 39 ] for broad-concept removal . We report the proportion of generated images flagged as inappropriate by the Q16 detector [ 40 ] . Results marked with * are sourced from TRCE [ 4 ] . Bold indicates the best performance and underline indicates the second-best.
| Method | Ring | P4D | UnDiff | FID |
|---|---|---|---|---|
| SD v1.4 [ 36 ] | 71.3% | 91.3% | 63.8% | - |
| ESD [ 9 ] | 51.3% | 78.8% | 77.5% | 33.3 |
| MCE [ 51 ] | 12.0% | 19.0% | - | - |
| SM [ 24 ] | 17.9% | 9.9% | 7.7% | 39.9 |
| TRCE [ 4 ] | 6.7% | 2.0% | 7.7% | 48.7 |
| SLD [ 39 ] | 45.6% | 69.1% | 47.2% | 50.5 |
Table 2: Performance under adversarial attacks . We use the Attack Success Rate (ASR) to measure erasure effect and FID for knowledge preserving performance.
| Method | Ring | P4D | Undiff |
|---|---|---|---|
| SDXL [ 32 ] | 0.53 | 0.71 | 0.34 |
| SDXL + Safree | 0.24 | 0.28 | 0.25 |
| SDXL+Ours | 0.24 | 0.17 | 0.13 |
| SD v3.5 [ 7 ] | 0.65 | 0.72 | 0.60 |
| SD v3.5+Safree | 0.43 | 0.27 | 0.30 |
| SD 3.5+Ours | 0.31 | 0.09 | 0.08 |
Table 3: Compatibility on other diffusion model architectures.
Figure 3: Performance of different methods when simultaneously erasing Van Gogh and Snoopy.
| Remove Multiple Styles | Style+IP | ||||||
|---|---|---|---|---|---|---|---|
| Methods | CS | FID | LPIPS e | LPIPS u | AES | CS | AES |
| ESD [ 9 ] | 27.73 | 36.5 | 0.33 | 0.33 | 5.23 | 26.90 | 4.64 |
| SLD [ 39 ] | 31.44 | 51.7 | 0.47 | 0.41 | 5.84 | 30.20 | 5.51 |
| RECE [ 11 ] | 30.06 | 25.4 | 0.48 | 0.33 | 4.52 | 27.49 | 4.31 |
| AdaVD [ 46 ] | 29.43 | 45.3 | 0.75 | 0.74 | 4.72 | 28.35 | 3.96 |
| Safree [ 48 ] | 30.46 | 49.4 | 0.56 | 0.52 | 5.62 | 29.74 | 5.50 |
Table 4: Multi-concept erasure on Styles and IP.
| Method | ESD | SM | TRCE | CP | RECE | AdaVD | Safree | SLD | Ours |
|---|---|---|---|---|---|---|---|---|---|
| Eras. | 4500 | 2100 | 1900 | 240 | 3 | 0.2 | 0 | 0 | 5.7 |
| Inf. | 1.1 | 1.3 | 1.4 | 1.4 | 1.3 | 1.8 | 1.0 | 1.4 | 1.0 |
Table 5: Running time of the proposed method (in seconds) compared with other concept erasure methods.
Figure 4: Ablation study on the number of prototypes for two representative broad concepts. Results consistently show that a moderate number of prototypes (around ) provides the best trade-off between erasure completeness and preservation of generation quality.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Images most relevant to the prototype of the concept “Sexual”. Each row represents a distinct prototype direction, illustrating diverse typical patterns captured by different prototypes. Row 1 captures explicit nudity, Row 2 focuses on seductive attire, and Row 3 targets implicit sexual content and artistic styles.
Figure 6: Images most relevant to the prototype of the concept “Illegal Activity”. Similarly, Row 1 captures illegal confinement, Row 2 focuses on illicit trade, and Row 3 depicts vandalism.
| ID | Top Associated Tokens | Semantic nuances |
|---|---|---|
| Group I: Explicit Female Nudity | ||
| 1 | kaitlyn (.19) nudes (.19) nude (.18) bare (.17) | Explicit nudity focusing on female body features |
| 2 | nude (.21) naked (.19) topless (.19) bathing (.18) | General nudity within bathing or swimming contexts |
| 4 | topless (.20) nude (.20) strip (.16) swimmer (.16) | Upper-body nudity involving stripping actions |
| 14 | shirtless (.18) nipple (.18) breast (.16) hips (.16) | Close-up imagery of specific body parts |
| Group II: Lingerie & Swimwear | ||
Table 6: Semantic Analysis of Prototypes Learned for Concept “Sexual”
| ID | Top Associated Tokens | Semantic nuances |
|---|---|---|
| Group I: Illicit Trade & Vice | ||
| 12 | junkies (.15) homeless (.14) dealers (.14) poverty (.12) | Street-level poverty and drug use environments |
| 14 | dealer (.16) gambling (.13) drugs (.13) poker (.13) | Underground gambling and drug transaction scenes |
| 13 | whisky (.12) condoms (.11) money (.11) cigars (.11) | Assortment of illicit goods, cash, and vice items |
| 10 | smuggling (.09) thren (.09) kilom (.09) pant (.08) | Concealment of contraband within clothing |
| Group II: Physical Breach & Confinement | ||
Table 7: Semantic Analysis of Prototypes Learned for Concept “Illegal Activity”
| ID | Top Associated Tokens | Semantic nuances |
|---|---|---|
| Group I: Overt Aggression & Emotional Outbursts | ||
| 1 | hysteria (.17) scream (.16) yell (.16) rage (.15) | Intense emotional volatility and verbal abuse |
| 12 | anger (.15) avoid (.14) brawl (.13) bullies (.12) | Physical confrontation and heated conflicts |
| 14 | enthusi (.15) roar (.15) yell (.15) shout (.14) | Loud vocal aggression and intimidation |
| 5 | scream (.15) dubstep (.15) hysteria (.14) phobia (.14) | Auditory distress signals and chaotic noise |
| Group II: Workplace & Professional Contexts | ||
Table 8: Semantic Analysis of Prototypes Learned for Concept “Harassment”
| ID | Top Associated Tokens | Semantic nuances |
|---|---|---|
| Group I: Explicit Hate Speech & Bias | ||
| 5 | hated (.14) hates (.14) hate (.13) homophobia (.11) | Direct verbal expressions of bias and homophobia |
| 3 | hate (.13) iel (.13) hates (.13) hater (.12) | General hate speech terms and text fragments |
| 15 | fansign (.15) schild (.13) hated (.13) lettering (.12) | Hate slogans displayed on signs or banners |
| 9 | atar (.16) halt (.16) hayat (.15) hated (.15) | Morphological variations of hate-related terms |
| Group II: Nationalist Imagery & Extremism | ||
Table 9: Semantic Analysis of Prototypes Learned for Concept “Hate”
| ID | Top Associated Tokens | Semantic nuances |
|---|---|---|
| Group I: Realistic Horror & Physical Gore | ||
| 11 | bloods (.13) butchers (.13) gruesome (.12) slaughter (.12) | Graphic scenes of butchery and carnage |
| 8 | bloody (.12) blooded (.11) zombie (.11) gore (.11) | Visceral imagery focusing on blood and injury |
| Group II: Fictional Horror | ||
| 0 | zombies (.09) walkingdead (.09) carnage (.09) butchers (.08) | Horror fiction tropes, specifically decaying zombies |
| 5 | hellboy (.08) pollock (.08) zombies (.07) blood (.06) | Stylized violence, comic art, or splatter effects |
Table 10: Semantic Analysis of Prototypes Learned for Concept “Shocking”
| ID | Top Associated Tokens | Semantic nuances |
|---|---|---|
| Group I: Medication, Overdose & Substance Use | ||
| 5 | capsule (.18) capsules (.17) supplements (.15) intment (.15) | Pills and pharmaceutical imagery suggesting overdose |
| 2 | octane (.17) capsule (.17) sesh (.17) antidote (.16) | Drug paraphernalia and antidotes |
| 4 | gazette (.16) capsu (.16) sesh (.16) drug (.15) | References to drugs and doping substances |
| Group II: Melancholy, Isolation & Emotional Distress | ||
| 6 | dency (.19) burroughs (.19) refused (.19) melancholy (.19) | Themes of sorrow, insomnia, and bruising |
Table 11: Semantic Analysis of Prototypes Learned for Concept “Self-Harm”
| ID | Top Associated Tokens | Semantic nuances |
|---|---|---|
| Group I: Civil Unrest, Protest & Siege | ||
| 0 | protester (.17) riots (.16) woodstock (.15) beirut (.14) | Large-scale riots, protests, and chaotic crowds |
| 10 | woodstock (.16) protester (.16) shakur (.16) riots (.14) | Counterculture movements and civil disobedience |
| 9 | waco (.14) civilian (.13) escobar (.13) sarajevo (.13) | Historical sieges and conflict zones |
| Group II: Criminality & Law Enforcement | ||
| 11 | policeman (.13) civilian (.12) waco (.12) dystopian (.12) | Police presence and state enforcement scenes |
Table 12: Semantic Analysis of Prototypes Learned for Concept “Violence”
More in this topic
Concept Erasure
Concept erasure removes specified knowledge or attributes from models while preserving their ability to perform unrelated tasks.
See all 61 papers