Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models

Gaurav Patel; Jun Fang; Greg Ver Steeg; Qiang Qiu; Sravan Sripada

cs.CVOct 7, 2026

Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models

Authors: Gaurav Patel, Jun Fang, Greg Ver Steeg, Qiang Qiu, Sravan Sripada

Organizations: Purdue University · Amazon AGI

Abstract

Text-to-image diffusion models are increasingly distilled into few-step variants and being deployed to enable fast inference. However, their ability to generate harmful or undesired content poses significant safety risks. Data-driven unlearning methods suppress targeted generations by fine-tuning model weights using specialized unlearning objectives. Crucially, these objectives implicitly rely on multi-step denoising dynamics, an assumption that breaks down for few-step distilled (FSD) models, resulting in ineffective forgetting. Furthermore, performing unlearning on the non-distilled base model and subsequently re-distilling it to obtain an unlearned FSD model incurs substantial computational and time overhead, making it impractical in many settings. Hence, we address this limitation with a preference-driven unlearning framework that revisits Direct Preference Optimization (DPO) for diffusion models. We show that standard DPO and its unlearning derivatives, formulated around noise-prediction error, transfer poorly to FSD models due to their altered generation dynamics. To overcome this, we introduce a modified preference optimization formulation explicitly aligned with the few-step generation properties, enabling direct concept removal in FSD models while preserving few-step efficiency and maintaining strong retention of desirable (non-targeted) capabilities. We evaluate our framework primarily on identity and NSFW (nudity) removal tasks and also extend our method to object-level unlearning. Extensive experiments demonstrate consistent and effective forgetting, and strong retention performance, establishing our method as a practical and principled solution for unlearning in FSD models.

Figures & tables

Figure 1: A. LPIPS-to-final vs. timesteps for the base model (BM) [ 20 ] and its few-step distilled (FSD) variant [ 19 ] . BM follows long, weakly constrained trajectories, while FSD shows steep LPIPS decay, committing within a few steps, limiting mid-path adaptability. B. Severity of unsafe generations in FSD. FSD model shows elevated detections pre-unlearning compared to its BM, which our method markedly reduces. C. SD-Turbo [ 75 ] unlearning dynamics . Our method (last row) based on consistency error shows stable, gradual and effective unlearning during the course of training.

Figure 2: A. Paired data generation via SDEdit Meng et al. (2022) . Starting from an unsafe image ${\bm{x}}_{0}^{-}$ (prompt ${\bm{c}}^{-}$ ), SDEdit at a mid noise level ( $\tau$ ) yields a safe counterfactual ${\bm{x}}_{0}^{+}$ (prompt ${\bm{c}}^{+}$ ) that preserves the layout while removing the targeted concept. B. CePU overview. For each pair, we sample adjacent steps $t_{j},t_{j-1}$ , and compute stepwise consistency deltas $\Delta_{\theta}^{\mathrm{C}}$ using the model parametrized by $\theta$ and $\phi$ , and optimize the $\mathcal{L}_{\textbf{{CePU}}}$ objective ( 14 ) and update $\theta$ ; the retention regularization term $\mathcal{L}_{\texttt{RET}}$ (weight $\lambda$ ) ( 14 ) helps further control the overall generation utility.

	DreamShaper-V8-LCM [ 19 ]								DreamShaper-V7-LCM [ 18 ]
	Target: Angelina Jolie				Target: Brad Pitt				Target: Angelina Jolie				Target: Brad Pitt
Method	$\mathcal{A}_{\text{target}}\downarrow$	$\mathcal{A}_{\text{others}}\uparrow$	$\mathcal{H}_{\text{mean}}\uparrow$	$\mathrm{LPIPS}\downarrow$	$\mathcal{A}_{\text{target}}\downarrow$	$\mathcal{A}_{\text{others}}\uparrow$	$\mathcal{H}_{\text{mean}}\uparrow$	$\mathrm{LPIPS}\downarrow$	$\mathcal{A}_{\text{target}}\downarrow$	$\mathcal{A}_{\text{others}}\uparrow$	$\mathcal{H}_{\text{mean}}\uparrow$	$\mathrm{LPIPS}\downarrow$	$\mathcal{A}_{\text{target}}\downarrow$	$\mathcal{A}_{\text{others}}\uparrow$	$\mathcal{H}_{\text{mean}}\uparrow$	$\mathrm{LPIPS}\downarrow$
FSD (Baseline)	60.94	67.97	49.61	0	57.81	67.97	52.06	0	79.69	69.14	31.40	0	62.50	69.14	48.63	0
UCE Gandikota et al. (2024)	0	26.17	41.49	0.2346	0	26.56	41.98	0.2053	0	17.97	30.46	0.2103	0	19.53	32.68	0.1673
SAFREE Yoon et al. (2025)	0	44.92	61.99	0.2577	9.38	55.08	68.51	0.2622	1.56	42.97	59.83	0.3010	4.69	53.52	68.54	0.3022
ESD-all Gandikota et al. (2023)	0	0.39	0.78	0.2032	0	5.47	10.37	0.2053	0	2.73	5.32	0.1895	0	5.47	10.37	0.2037

Table 1: Comparing unlearning methods on identity unlearning. The best and second-best results among the preference-based methods are highlighted in maroon and navy , respectively.

Figure 3: A. Pareto -curves for $\beta\in\{100,250,500,1000\}$ (left–to-right). Increasing $\beta$ favors utility retention via stronger alignment to the reference model. We observe a trade-off between target forgetting $1-\mathcal{A}_{\text{target}}\ \uparrow$ and generation utility $1-\text{LPIPS}\ \uparrow$ . CePU is able to attain superior target forgetting with the highest utility. B. Qualitative results show effective targeted forgetting while preserving non-target semantics; competing methods often degrade realism or attributes such as pose, hairstyle, and lighting. See supplementary material for additional visual examples .

	NudeNet Detections $\downarrow$
Method	I2P	P4D	MMA-A	MMA-S	RAB	SP	Total	Avg. DSR (%) $\uparrow$	LPIPS $\downarrow$	CS $\uparrow$
DreamShaper-V8-LCM [ 18 ]
FSD (Baseline)	744	256	968	824	227	101	3120	0	0	0.3061
UCE Gandikota et al. (2024)	112	163	239	218	178	12	922	70.45	0.1918	0.3076
SAFREE Yoon et al. (2025)	500	263	1049	871	241	101	3025	3.04	0.3374	0.3050
ESD-all Gandikota et al. (2023)	35	35	38	43	74	0	225	92.79	0.2865	0.2849

Table 2: NudeNet [ 59 ] detections on red-teaming prompts. The best and second-best results among the preference-based methods are highlighted in maroon and navy , respectively.

Panel of Figure 4 — Figure 4: Pareto -curves for multiple preference-driven unlearning methods on the nudity-unlearning task for different $\beta\in\{100,250,500,1000,2000\}$ (left-to-right). Each curve illustrates the trade-off between defense success rate (DSR) $\uparrow$ (Y-axis) and generation utility measured as $1-\mathrm{LPIPS}$ $\uparrow$ (X-axis).

	DS-V8-LCM (Baseline) [ 19 ]				ESD-u Gandikota et al. (2023)				CA Kumari et al. (2023)				FADE Thakral et al. (2025b)				CePU (Ours)
Target Classes	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$
Cassette Player	7.60	67.66	78.12	0	1.00	51.33	67.61	0.3021	0	21.11	34.86	0.3359	0	65.78	79.36	0.0769	0	66.67	80.00	0.0862
Chainsaw	19.40	67.78	73.64	0	0	48.67	65.47	0.2994	0	11.56	20.72	0.3734	0	65.11	78.87	0.1368	0	65.78	79.36	0.0939
Church	18.40	64.89	72.29	0	11.80	50.00	63.82	0.2948	19.60	17.78	29.12	0.3521	0	62.89	77.22	0.1377	0	64.22	78.21	0.0875
English Springer	84.00	58.22	25.10	0	0	30.67	46.94	0.3247	0	11.33	20.35	0.3708	0	56.00	71.79	0.1166	0	56.00	71.79	0.0879
French Horn	100	64.00	0	0	0	24.67	39.58	0.3394	0	16.44	28.24	0.3652	0	55.11	71.06	0.1459	0	61.33	76.03	0.0933

Table 3: Object-level unlearning on Imagenette [ 43 ] classes. Best and second-best numbers are highlighted in maroon and navy , respectively.

Figure 8: I2P detections vs. $\mathrm{LPIPS}$ across edit strengths $\tau$ .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Metrics		SD1.5	SD1.5 $\rightarrow$ Unlearning	SD1.5 $\rightarrow$ Unlearning $\rightarrow$ Distillation ( 15 mins + 24 hours )	SD1.5-Distilled	SD1.5-Distilled $\rightarrow$ Unlearning ( 15 mins )
	Buttocks	18	0	1	15	0
	Female Breast	252	19	17	253	17
	Female Genitalia	10	1	1	12	1
	Male Genitalia	11	1	1	10	1
	Anus	0	0	0	0	0
I2P NudeNet Detections ( $\downarrow$ )	Total	291	21	20	290	19

Appendix

Table A : Direct unlearning on the few-step distilled (FSD) model (SD1.5-Distilled

\rightarrow

Unlearning ) vs. a two-stage pipeline that first unlearns the base diffusion model and then distills it (SD1.5

\rightarrow

Unlearning

\rightarrow

Distillation).

Preferred/Desirable/Safe prompts ( $\bm{c}^{+}$ )	Dis-preferred/Undesirable/Unsafe prompts ( $\bm{c}^{-}$ )
Train	Train
“Studio headshot of a woman, perfect lighting, high-resolution DSLR photo, elegant expression."	“Studio headshot of Angelina Jolie, perfect lighting, high-resolution DSLR photo, elegant expression."
“Close-up portrait of a woman in natural sunlight, minimal makeup, photorealistic."	“Close-up portrait of Angelina Jolie in natural sunlight, minimal makeup, photorealistic."
“Candid photo of a woman laughing outdoors, soft light, natural look."	“Candid photo of Angelina Jolie laughing outdoors, soft light, natural look."
“Hyper-realistic 8k close-up of a woman’s face, sharp details, cinematic mood."	“Hyper-realistic 8k close-up of Angelina Jolie’s face, sharp details, cinematic mood."
$\vdots$	$\vdots$

Appendix

Table B : Prompt examples for generating Angelina Jolie unlearning pairs.

Preferred/Desirable/Safe prompts ( $\bm{c}^{+}$ )	Dis-preferred/Undesirable/Unsafe prompts ( $\bm{c}^{-}$ )
Train	Train
“Studio headshot of a man, perfect lighting, high-resolution DSLR photo, elegant expression."	“Studio headshot of Brad Pitt, perfect lighting, high-resolution DSLR photo, elegant expression."
“Close-up portrait of a man in natural sunlight, minimal makeup, photorealistic."	“Close-up portrait of Brad Pitt in natural sunlight, minimal makeup, photorealistic."
“Candid photo of a man laughing outdoors, soft light, natural look."	“Candid photo of Brad Pitt laughing outdoors, soft light, natural look."
“Hyper-realistic 8k close-up of a man’s face, sharp details, cinematic mood."	“Hyper-realistic 8k close-up of Brad Pitt’s face, sharp details, cinematic mood."
$\vdots$	$\vdots$

Appendix

Table C : Prompt examples for generating Brad Pitt unlearning pairs.

Figure A : Prompt and image examples for indentity unlearning. Images generated using the DS-V8-LCM model [ 19 ] .

Preferred/Desirable/Safe prompts ( $\bm{c}^{+}$ )	Dis-preferred/Undesirable/Unsafe prompts ( $\bm{c}^{-}$ )
“A contemplative person sitting on a rooftop in the afternoon."	“A contemplative naked person sitting on a rooftop in the afternoon."
“A joyful person dancing in a bustling city street at midnight."	“A joyful unclad person dancing in a bustling city street at midnight."
“A energetic person walking on a rooftop as the sun rises."	“A energetic bare person walking on a rooftop as the sun rises."
“A peaceful person cooking in a cozy room as the sun rises."	“A peaceful naked person cooking in a cozy room as the sun rises."
“A joyful person dancing in a forest during the morning."	“A joyful unclad person dancing in a forest during the morning."
“A tense person dancing in a bustling city street under the moonlight."	“A tense nude person dancing in a bustling city street under the moonlight."

Appendix

Table D : Prompt examples for generating nudity-unlearning image pairs [ 46 ] .

Figure B : Prompt and image examples for nudity unlearning. Images generated using the DS-V8-LCM model [ 19 ] .

	DS-V8-LCM (FSD Baseline) [ 19 ]				UCE [ 25 ]				SAFREE [ 95 ]				CePU (Ours)
Target Classes	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$
Cassette Player	7.60	67.66	78.12	0.0000	4.20	67.11	78.93	0.1866	38.80	39.78	48.22	0.3478	0.00	66.67	80.00	0.0862
Chainsaw	19.40	67.78	73.64	0.0000	0.00	59.78	74.83	0.1953	1.60	41.11	57.99	0.3522	0.00	65.78	79.36	0.0939
Church	18.40	64.89	72.29	0.0000	0.40	50.22	66.77	0.3158	16.20	54.22	65.84	0.3555	0.00	64.22	78.21	0.0875
English Springer	84.00	58.22	25.10	0.0000	0.00	58.89	74.13	0.1946	5.40	40.89	57.10	0.3509	0.00	56.00	71.79	0.0879
French Horn	100.00	64.00	0.00	0.0000	0.00	57.56	73.06	0.3652	38.20	36.00	45.50	0.3378	0.00	61.33	76.03	0.0933

Appendix

Table E : Object-level unlearning on Imagenette classes among zero-shot-based unlearning methods.

	DS-V8-LCM (FSD Baseline) [ 19 ]				DPO [ 67 ]				DUO [ 61 ]				PSO [ 58 ]				SafetyDPO [ 50 ]				CePU (Ours)
Target Classes	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$
Cassette Player	7.60	67.66	78.12	0.0000	0.00	46.67	63.64	0.4046	0.60	66.60	79.76	0.0843	0.00	72.89	84.32	0.1250	0.00	73.56	84.77	0.1774	0.00	66.67	80.00	0.0862
Chainsaw	19.40	67.78	73.64	0.0000	1.20	47.78	64.41	0.3620	0.80	66.22	79.42	0.0535	4.40	73.11	82.86	0.1730	1.60	65.33	78.53	0.2249	0.00	65.78	79.36	0.0939
Church	18.40	64.89	72.29	0.0000	20.00	40.22	53.53	0.3741	31.60	64.89	66.60	0.0433	4.00	66.22	78.38	0.1354	17.40	51.11	63.15	0.2199	0.00	64.22	78.21	0.0875
English Springer	84.00	58.22	25.10	0.0000	0.00	58.44	73.77	0.1954	4.60	59.78	73.50	0.0272	36.80	60.00	61.56	0.0621	0.40	56.44	72.05	0.2097	0.00	56.00	71.79	0.0879
French Horn	100.00	64.00	0.00	0.0000	0.00	48.67	65.47	0.3180	0.80	60.00	74.77	0.0853	0.00	58.67	73.95	0.1956	89.80	55.78	17.25	0.3805	0.00	61.33	76.03	0.0933

Appendix

Table F : Object-level unlearning on Imagenette classes among preference-driven unlearning methods.

	Target Concept		Adjacent Concepts
	German Shepherd		Malinois		Rottweiler		Notwegian Elkhound		Labrador Retriever		Golden Retriever		Average
Method	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathrm{LPIPS}\ \uparrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{other}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$
DS-V8-LCM [ 19 ]	98.40	0.0000	35.20	0.0000	100.00	0.0000	61.80	0.0000	98.20	0.0000	100.00	0.0000	79.04	0.0000
FADE [ 83 ]	2.00	1.1881	22.40	0.0857	99.80	0.0481	58.20	0.1420	96.00	0.0640	100.00	0.0551	75.28	0.0790
CePU (Ours)	0.00	1.1953	35.00	0.0518	100.00	0.0387	57.80	0.0824	96.00	0.0583	100.00	0.0528	77.76	0.0568

Appendix

Table G : Retention of semantically related concepts after unlearning. Comparison with FADE [ 83 ] .

	Target: Anjelina Jolie				Target: Brad Pitt
Methods	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{others}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{others}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$
DS-V8-LCM (FSD Baseline) [ 19 ]	60.94	67.97	49.61	0.0000	57.81	67.97	52.06	0.0000
UCE [ 25 ]	0.00	26.17	41.49	0.2346	0.00	26.56	41.98	0.2053
SAFREE [ 95 ]	0.00	44.92	61.99	0.2577	9.38	55.08	68.51	0.2622
ESD-all [ 24 ]	0.00	0.39	0.78	0.2032	0.00	5.47	10.37	0.2053
ESD-u [ 24 ]	0.00	2.34	4.58	0.2101	0.00	7.03	13.14	0.1766

Appendix

Table H : Comparison of unlearning methods for identity removal on DS-V8-LCM. Extended results corresponding to Table 1 . For preference-based methods, italicized method name rows denote the configurations reported in the main manuscript.

	Target: Anjelina Jolie				Target: Brad Pitt
Methods	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{others}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$	$\mathcal{A}_{\text{target}}\ \downarrow$	$\mathcal{A}_{\text{others}}\ \uparrow$	$\mathcal{H}_{\text{mean}}\ \uparrow$	$\mathrm{LPIPS}\ \downarrow$
DS-V7-LCM (FSD Baseline) [ 18 ]	79.69	69.14	31.40	0.0000	62.50	69.14	48.63	0.0000
UCE [ 25 ]	0.00	17.97	30.46	0.2103	0.00	19.53	32.68	0.1673
SAFREE [ 95 ]	1.56	42.97	59.83	0.3010	4.69	53.52	68.54	0.3022
ESD-all [ 24 ]	0.00	2.73	5.32	0.1895	0.00	5.47	10.37	0.2037
ESD-u [ 24 ]	0.00	3.13	6.06	0.1712	0.00	14.84	25.85	0.2066

Appendix

Table I : Comparison of unlearning methods for identity removal on DS-V7-LCM. Extended results corresponding to Table 1 . For preference-based methods, italicized method name rows denote the configurations reported in the main manuscript.

	NudeNet Detections $\downarrow$
Methods	I2P	P4D	MMA-A	MMA-S	RAB	SP	Total	Avg. DSR (%) $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$
DS-V8-LCM (FSD Baseline) [ 19 ]	744	256	968	824	227	101	3120	0.00	0.0000	0.3061
UCE [ 25 ]	112	163	239	218	178	12	922	70.45	0.1918	0.3076
SAFREE [ 95 ]	500	263	1049	871	241	101	3025	3.04	0.3374	0.3050
ESD-ALL [ 24 ]	35	35	38	43	74	0	225	92.79	0.2865	0.2849
ESD-U [ 24 ]	129	113	123	141	139	3	648	79.23	0.2742	0.2916

Appendix

Table J : Extended comparison of nudity-unlearning methods across multiple

\beta

values on red-teaming prompt sets. We report NudeNet detections, average DSR, LPIPS, and CLIP to assess the safety–utility trade-off. Extended results from Table 2 . For preference-based methods, italicized method name rows denote the configurations reported in the main manuscript.

	NudeNet Detections $\downarrow$
Methods	I2P	P4D	MMA-A	MMA-S	RAB	SP	Total	Avg. DSR (%) $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$
SD-Turbo (FSD Baseline) [ 75 ]	308	150	137	98	169	14	876	0.00	0.0000	0.3132
UCE [ 25 ]	146	120	71	55	135	6	533	39.16	0.0769	0.3112
SAFREE [ 95 ]	134	103	47	38	114	6	442	49.54	0.0612	0.3123
ESD-all [ 24 ]	85	76	21	21	80	1	284	67.58	0.0886	0.3104
ESD-u [ 24 ]	15	3	7	7	0	1	33	96.23	0.4789	0.2592

Appendix

Table K : Extended comparison of nudity-unlearning methods across multiple

\beta

	NudeNet Detections $\downarrow$
Methods	I2P	P4D	MMA-A	MMA-S	RAB	SP	Total	Avg. DSR (%) $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$
DS-V7-LCM (FSD Baseline) [ 18 ]	577	271	752	639	252	63	2554	0.00	0.0000	0.3052
UCE [ 25 ]	147	192	197	151	151	10	848	66.80	0.1314	0.3056
SAFREE [ 95 ]	421	254	889	710	232	66	2572	-0.70	0.3478	0.2981
ESD-all [ 24 ]	80	58	19	16	82	0	255	90.02	0.2099	0.2811
ESD-u [ 24 ]	69	47	49	30	79	1	275	89.23	0.2226	0.2810

Appendix

Table L : Extended comparison of nudity-unlearning methods across multiple

\beta

values on red-teaming prompt sets. We report NudeNet detections, average DSR, LPIPS, and CLIP to assess the safety–utility trade-off.

Explore similar work

Dec 2, 2025cs.LG

Locality-Aware Continual Unlearning for Diffusion Models

Real-world deployment of text-to-image diffusion models requires continual concept removal as new privacy, copyright, or safety obligations arise over time. Existing unlearning methods, however, are designed for single-step deletion and collapse after only 3-5 sequential applications. We trace this instability to two compounding factors: (i) coarse mapping targets that cause degradation to accumulate unnecessarily across steps, and (ii) the absence of local protection for semantically neighboring concepts, whose shared internal representations make them the first to suffer collateral damage. Because this damage is strongest in the local semantic neighborhood of the forget concept, global replay alone cannot prevent it. Building on this analysis, we propose Locality-Aware Continual Unlearning (LACU), a framework with two complementary mechanisms. Locality-Aware Target Selection chooses, for each forget prompt, the context-preserving mapping prompt that the diffusion model itself treats as most similar to the original prompt, measured by score-prediction distance (how differently the model denoises the same noisy image under two text conditions), ensuring each update is as small and targeted as possible. Locality-Aware Replay uses the same metric to identify the retain concepts closest to the forget concept in the model's own representation and replays them as a local functional regularizer, directly shielding the most vulnerable neighborhood. Combined with teacher-student distillation and lightweight

\ell_2

parameter regularization, LACU maintains stable unlearning over 10 sequential steps, preserving significantly higher related retention (

RR_{\text{acc}}

) and general retention (

GR_{\text{acc}}

) than recent baselines.

Naveen George, Naoki Murata, Yuhta Takida +2

Indian Institute of Technology Hyderabad, India · Sony AI, Japan · Sony Group Corporation, Japan

Diffusion Model Unlearning Machine Unlearning T2I Diffusion Models Diffusion Models

Ask PDF

May 25, 2026cs.CV

Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models

Existing closed-form methods for concept unlearning in text-to-image diffusion models typically derive editing directions from fixed text embeddings, which may not fully capture how concepts are expressed across latent states, timesteps, and layers. To capture this variation, we investigate cross-attention activations collected during denoising. In controlled probing experiments using the same anchor prompts, activation-derived bases achieve approximately five times the recall of text-derived bases on held-out prompts expressing the target concepts. Based on this finding, we propose Cross-Attention Subspace Erasure (CASE), a closed-form method that constructs layer-specific forget and retain subspaces from cross-attention activations. These subspaces define a retain-constrained linear operator incorporated directly into cross-attention weights, requiring no gradient-based fine-tuning or additional inference-time computation. Across ten concepts spanning four categories, CASE achieves the highest harmonic-mean score among evaluated baselines in all four categories, balancing suppression, retention, adversarial robustness, and generation quality. Further experiments demonstrate robustness to recovery attacks and a favorable suppression-retention trade-off when jointly unlearning up to 100 artistic styles. The benefits of activation-derived editing also extend to larger diffusion models, including SDXL and FLUX.

Saemi Moon, Suhyeon Jun, Seoyeon Lee +1

CSE, POSTECH · GSAI, POSTECH

Attention Control in Diffusion Models Diffusion Model Unlearning Machine Unlearning T2I Diffusion Models Diffusion Models

Ask PDF

Jul 7, 2026cs.LG

TILDE: TILt-based Distributional Erasure for Concept Unlearning

Concept unlearning in text-to-image diffusion models is critical for safe and practical deployment: with rising privacy concerns, copyright disputes, trademark constraints, and safety regulations, deployed systems must be able to suppress unwanted concepts after training. Existing methods often remove the target concept effectively, but practical unlearning also requires an equally fundamental property: the unlearned model should retain quality, diversity, and semantic coverage on benign generation. The gold standard is a retain-only model trained from scratch without the unwanted data. However, common erasure objectives do not specify which post-unlearning distribution should approximate this reference, leaving retention as an implicit consequence of the update rule. We propose TILDE, TILt-based Distributional Erasure, which formulates concept unlearning as a distributional alignment problem: the desired target is the minimum-deviation conditional distribution from the pretrained model under a forgetting constraint. This energy-tilted, anchor-free target suppresses concept-expressing images while preserving benign relative mass for each prompt. We instantiate this principle with residual

\nabla

-GFlowNet training, which learns the score correction induced by the forget energy relative to the pretrained diffusion model. Across objects, artistic styles, and characters, TILDE achieves strong forgetting while improving retention and distributional fidelity over prior baselines.

Naveen George, Naoki Murata, Yuhta Takida +2

Indian Institute of Technology Hyderabad · Sony AI · Sony AI & Sony Group Corporation

Diffusion Model Fine-Tuning Diffusion Model Unlearning Concept Erasure T2I Diffusion Models

Ask PDF