PROBE: Diagnosing Residual Concept Capacity in Erased Text-to-Video Diffusion Models

Yiwei Xie; Zheng Zhang; Ping Liu

cs.CVMar 23, 2026

PROBE: Diagnosing Residual Concept Capacity in Erased Text-to-Video Diffusion Models

Authors: Yiwei Xie, Zheng Zhang, Ping Liu

Organizations: School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China · Department of Computer Science and Engineering, University of Nevada, Reno, NV, USA

Abstract

Concept erasure techniques for text-to-video (T2V) diffusion models report substantial suppression of sensitive content, yet current evaluation is limited to checking whether the target concept is absent from generated frames, treating output-level suppression as evidence of representational removal. We introduce PROBE, a diagnostic protocol that quantifies the reactivation potential of erased concepts in T2V models. With all model parameters frozen, PROBE optimizes a lightweight pseudo-token embedding through a denoising reconstruction objective combined with a novel latent alignment constraint that anchors recovery to the spatiotemporal structure of the original concept. We make three contributions: (1) a multi-level evaluation framework spanning classifier-based detection, semantic similarity, temporal reactivation analysis, and human validation; (2) systematic experiments across three T2V architectures, three concept categories, and three erasure strategies revealing that all tested methods leave measurable residual capacity whose robustness correlates with intervention depth; and (3) the identification of temporal re-emergence, a video-specific failure mode where suppressed concepts progressively resurface across frames, invisible to frame-level metrics. These findings suggest that current erasure methods achieve output-level suppression rather than representational removal. We release our protocol to support reproducible safety auditing. Our code is available at https://github.com/YiweiXie/PRObingBasedEvaluation.

Figures & tables

Fig. 1 : Overview of the PROBE diagnostic framework.

Origin

NegPrompt

SAFREE

T2VUnlearning

Object Class

Cassette player

9.41

39.71

.175

1.76 / 2.65

21.76 / 25.00

.120 / .139

0.59 / 2.65

18.82 / 13.53

.104 / .112

0.00 / 1.18

8.82 / 19.41

.092 / .102

Chain saw

42.35

63.82

.146

18.24 / 22.06

41.76 / 44.41

.118 / .119

5.59 / 6.76

14.71 / 17.35

.110 / .117

0.00 / 0.00

3.24 / 1.47

.091 / .087

Church

59.70

77.65

.160

24.12 / 30.00

47.65 / 60.00

.147 / .152

3.53 / 6.47

15.88 / 37.35

.123 / .144

19.12 / 21.76

51.18 / 45.00

.141 / .141

Eng. springer

10.59

16.76

.141

11.47 / 11.76

24.12 / 32.35

.134 / .142

0.00 / 3.82

6.18 / 8.82

.088 / .097

5.59 / 6.67

7.94 / 11.18

.128 / .133

French horn

80.29

93.24

.188

24.41 / 29.71

50.59 / 66.18

.121 / .208

7.35 / 10.88

26.18 / 22.06

.171 / .177

1.47 / 3.53

10.00 / 15.59

.148 / .158

TABLE I : Object erasure results on CogVideoX-2B across three metrics (Erased / After PROBE). T1: Top-1 accuracy (%), T5: Top-5 accuracy (%), CL: CLIP similarity. The highest post-PROBE recovery per row per metric is in bold .

Fig. 2 : Temporal reactivation curve of parachute across different erasure strategies on CogX-2B.

Metrics	Origin	NegPrompt	SAFREE	T2VUnlearning
Mean Var	0.1834 0.0011	0.0819 / 0.1080 0.0009 / 0.0012	0.0514 / 0.0654 0.0007 / 0.0011	0.0137 / 0.0183 0.0002 / 0.0003

TABLE II : Temporal reactivation statistics for object erasure on CogX-2B (Erased / After PROBE).

Model	Origin	NegPrompt	SAFREE	T2VUnlearning
CogX-2B	56.14	42.82 / 56.16	35.16 / 56.16	19.63 / 28.90
CogX-5B	56.08	47.08 / 61.76	38.48 / 58.32	11.12 / 12.40
Wan2.2-5B	67.28	56.12 / 65.28	45.76 / 51.68	18.64 / 31.28

TABLE III : Nudity rate (%) across three T2V models (Erased / After PROBE).

Fig. 3 : Temporal reactivation curve of nudity across different erasure strategies on CogX-2B.

Exposed Class	Origin	NegPrompt	SAFREE	T2VUnlearning
CogVideoX-2B
Female genitalia	213	91 / 111	82 / 130	48 / 48
Male genitalia	13	14 / 17	86 / 33	56 / 69
Female breast	1003	631 / 910	397 / 536	377 / 615
Male breast	75	70 / 43	78 / 51	8 / 25
Buttocks	650	449 / 925	470 / 700	168 / 223

TABLE IV : Class-level NudeNet detections on CogVideoX-2B and CogVideoX-5B (Erased / After PROBE).

Model	Origin	NegPrompt	SAFREE	T2VUnlearning
CogX-2B	0.1597	0.1421 / 0.1522	0.1417 / 0.1516	0.1309 / 0.1385
CogX-5B	0.1422	0.1334 / 0.1444	0.1285 / 0.1483	0.1039 / 0.1036
Wan2.2-5B	0.1333	0.1286 / 0.1395	0.1159 / 0.1303	0.1029 / 0.1188
Average	0.1451	0.1347 / 0.1454	0.1287 / 0.1434	0.1126 / 0.1203

TABLE V : CLIP-based similarity for NSFW concepts across three T2V models (Erased / After PROBE).

Model	Metrics	Origin	NegPrompt	SAFREE	T2VUnlearning
CogX-2B	Mean Var	0.3201 0.0003	0.2566 / 0.3146 0.0001 / 0.0002	0.2163 / 0.2497 0.0001 / 0.0001	0.1212 / 0.1851 0.0001 / 0.0000
CogX-5B	Mean Var	0.3387 0.0000	0.2891 / 0.3550 0.0001 / 0.0001	0.2464 / 0.3625 0.0001 / 0.0001	0.0824 / 0.0994 0.0000 / 0.0000
Wan-5B	Mean Var	0.4437 0.0001	0.3660 / 0.4171 0.0001 / 0.0001	0.3003 / 0.3370 0.0000 / 0.0000	0.1364 / 0.2059 0.0000 / 0.0000

TABLE VI : Temporal reactivation statistics for nudity erasure (Erased / After PROBE).

	Metrics	Origin	NegPrompt	SAFREE	T2VUnlearning
CogX-2B	Avg. Score Std_Dev	1.150 0.484	1.410 / 1.490 0.367 / 0.433	1.040 / 0.950 0.629 / 0.331	0.550 / 0.300 0.448 / 0.343
CogX-2B	Fleiss’ $\kappa$	0.2577
CogX-5B	Avg. Score Std_Dev	1.440 0.613	1.420 / 1.310 0.410 / 0.507	1.070 / 1.570 0.579 / 0.313	0.490 / 0.540 0.595 / 0.642
CogX-5B	Fleiss’ $\kappa$	0.3457
Wan-5B	Avg. Score Std_Dev	1.250 0.803	1.520 / 1.560 0.555 / 0.433	0.700 / 1.270 0.751 / 0.598	0.770 / 1.060 0.442 / 0.409
Wan-5B	Fleiss’ $\kappa$	0.3362

TABLE VII : Human validation of nudity erasure (Erased / After PROBE).

Model	Identity	Origin	Erased	After PROBE
CogX-2B	Obama	0.4653	0.2684	0.2834
CogX-2B	Trump	0.6503	0.2621	0.2966
CogX-5B	Obama	0.6548	0.0600	0.0945
CogX-5B	Trump	0.6233	0.1232	$-$ 0.0066

TABLE VIII : ID-similarity for celebrity identity recovery on CogVideoX with T2VUnlearning.

Fig. 4 : Temporal reactivation curve of identity on CogX-2B.

Model	Metrics	Origin	T2VUnlearning
CogX-2B	Mean Var	0.5578 0.0008	0.2653 / 0.2900 0.0004 / 0.0001
CogX-5B	Mean Var	0.6391 0.0004	0.0916 / 0.0440 0.0004 / 0.0002

TABLE IX : Temporal reactivation statistics for identity erasure (Erased / After PROBE).

Model	T2VUnlearning	P4D-K	PROBE
CogX-2B	19.63	20.45	28.90

TABLE X : Comparison of PROBE with P4D-K on CogX-2B under T2VUnlearning. Nudity rate (%).

Model	Origin	NegPrompt	SAFREE	T2VUnlearning
CogX-2B	36.45	22.14 / 26.53	17.44 / 32.27	11.37 / 13.95

TABLE XI : Cross-dataset evaluation on Ring-A-Bell. Nudity rate (%) on CogX-2B (Erased / After PROBE).

Transfer Direction	Erased Baseline	After Transfer
CogX-2B $\rightarrow$ Wan2.2-5B	18.64	15.80
CogX-5B $\rightarrow$ Wan2.2-5B	18.64	17.24
CogX-2B $\rightarrow$ CogX-5B	11.12	12.92
Wan2.2-5B $\rightarrow$ CogX-5B	11.12	11.00

TABLE XII : Cross-model transferability. Nudity rate (%) under T2VUnlearning.

Transfer Direction	Erased Baseline	After Transfer
T2VUnlearning $\rightarrow$ NegPrompt	42.82	60.02
T2VUnlearning $\rightarrow$ SAFREE	35.16	41.63

TABLE XIII : Cross-method transferability on CogX-2B. Nudity rate (%).

Metric	Origin	NegPrompt	SAFREE	T2VUnlearning
Imaging Quality	0.467	0.477 / 0.519	0.446 / 0.460	0.380 / 0.396
Aesthetic Quality	0.460	0.481 / 0.490	0.459 / 0.464	0.424 / 0.433

TABLE XIV : Visual quality on CogX-2B (Erased / After PROBE). Higher is better.

Model (Concept)	Erased	w/o $\mathcal{L}_{\text{align}}$	w/ $\mathcal{L}_{\text{align}}$
CogX-2B (Object)	7.44	8.56	10.32
CogX-2B (Nudity)	19.63	22.16	28.90
CogX-5B (Nudity)	11.12	11.36	12.40
Wan2.2-5B (Nudity)	18.64	27.92	31.28

TABLE XV : Ablation studies on T2VUnlearning. (Left) Reactivation rates (%) w/ and w/o the alignment loss

\mathcal{L}_{\text{align}}

. (Right) Nudity reactivation rates (%) under varying penalty weights

\lambda

	Erased	10	50	100	200
Reference size	19.63	25.60	25.92	28.90	23.47
	Erased	3 tokens	5 tokens	10 tokens
Token count	19.63	27.18	28.90	25.20

TABLE XVI : Sensitivity to reference data size and pseudo-token count on CogX-2B under T2VUnlearning (nudity rate, %).

CogX-2B_Object erasure
Metrics	Origin	NegPrompt	SAFREE	T2VUnlearning
Avg. Score	1.340	1.215 / 0.835	0.660 / 0.790	0.695 / 0.515
Std_Dev	0.593	0.593 / 0.627	0.620 / 0.548	0.670 / 0.545
Fleiss’ $\kappa$	0.3391
Identity erasure
	Metrics	Origin	T2VUnlearning

TABLE XVII : Human validation of object and identity erasure on CogX-2B (Erased / After PROBE).

Panel of Figure 22 — Fig. 5 : Attention heatmaps across different erasure methods. Rows: original model, erased, PROBE. Columns: different object categories. PROBE recovers coherent spatial attention patterns even after weight-space erasure.

Fig. 6 : Temporal reactivation curve of nudity across different erasure strategies on CogX-5B.

Fig. 7 : Temporal reactivation curve of nudity across different erasure strategies on Wan2.2-5B.

Fig. 8 : Temporal reactivation curve of identity erasure on CogX-5B.

Setting	Origin	$\lambda=0$	$\lambda=1$
T2VUnlearning	78.61 $\rightarrow$ 16.67	27.78	20.45

TABLE XVIII : Nudity rate (%) on HunyuanVideo under T2VUnlearning and PROBE with different

\lambda

values.

Diffusion Model Unlearning

Diffusion model unlearning removes specified concepts or training examples from generative models while preserving unrelated generation.

See all 72 papers

FADE: Frame-Aware Diffusion-Transformer-based Multi-Concept Erasure for Video Unlearning
Oct 2, 2026Yuchen Li, Kaiyuan Deng, Chaoran Feng +2Diffusion Model Unlearning Machine Unlearning
Peking University · University of Arizona
EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders
Sep 3, 2026Xinghao Wang, Dong Li, Wei Yu +5Video Diffusion Models Sparse Autoencoders
University of Science and Technology of China, Hefei, China · HiDream.ai Inc. · Anhui Province Key Laboratory of Digital Security, China
Motion Concept Unlearning in Video Diffusion Models
Sep 29, 2026Ping Liu, Chi ZhangVideo Diffusion Models Diffusion Transformer
Computer Science and Engineering University of Nevada, Reno Reno, NV, USA · Department of Mathematics National University of Singapore Singapore

Oct 2, 2026J/K move · Enter open · S save

FADE: Frame-Aware Diffusion-Transformer-based Multi-Concept Erasure for Video Unlearning

Yuchen Li, Kaiyuan Deng, Chaoran Feng, Zhenyu Tang+1

Peking University · University of Arizona

Diffusion Model Unlearning Machine Unlearning T2V Generation

Open paper AskPDF arXiv

Text-to-video (T2V) diffusion models can reproduce copyrighted, violent, or explicit content, which motivates concept erasure: removing designated concepts from a pretrained model while preserving its behavior on everything else. Existing T2V erasure methods leave two problems open. Their frame-agnostic suppression can leave isolated frames in which an erased concept resurfaces, a frame-reactivation gap that clip-level averages obscure; and they are usually evaluated with one target concept or category at a time. We propose Frame-Aware Diffusion Erasure (FADE), a multi-concept video unlearning framework. FADE first applies a joint closed-form key/value edit that suppresses all target concepts, then trains per-concept frame-aware low-rank adapters whose strength is gated by the frame index and the denoising timestep to remove residual per-frame leakage. Each adapter is trained with the other targets' prompts as hard negatives, which keeps the concept-specific components of different adapters well separated, and a similarity-based soft router combines the adapters according to the prompt. With 16 concepts (objects, artistic styles, and nudity) erased from a single Wan2.1-T2V-1.3B backbone, FADE reduces the residual accuracy on the object benchmark to 4.9%, against 15.5% for the strongest of eight baselines, while keeping the VBench average within 0.9% of the unedited model. The ranking is unchanged under a VLM judge and a blinded human study, and the advantage over the strongest baseline carries over to prompts that combine several erased concepts, to 30 simultaneously erased celebrity identities, and to Wan2.1-T2V-14B, CogVideoX-2B, and HunyuanVideo-1.5.

Abstract

Diffusion Model Unlearning

Explore similar work

FADE: Frame-Aware Diffusion-Transformer-based Multi-Concept Erasure for Video Unlearning

EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders

Motion Concept Unlearning in Video Diffusion Models