PROBE: Diagnosing Residual Concept Capacity in Erased Text-to-Video Diffusion Models
Authors: Yiwei Xie, Zheng Zhang, Ping Liu
Organizations: School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China · Department of Computer Science and Engineering, University of Nevada, Reno, NV, USA
Concept erasure techniques for text-to-video (T2V) diffusion models report substantial suppression of sensitive content, yet current evaluation is limited to checking whether the target concept is absent from generated frames, treating output-level suppression as evidence of representational removal. We introduce PROBE, a diagnostic protocol that quantifies the reactivation potential of erased concepts in T2V models. With all model parameters frozen, PROBE optimizes a lightweight pseudo-token embedding through a denoising reconstruction objective combined with a novel latent alignment constraint that anchors recovery to the spatiotemporal structure of the original concept. We make three contributions: (1) a multi-level evaluation framework spanning classifier-based detection, semantic similarity, temporal reactivation analysis, and human validation; (2) systematic experiments across three T2V architectures, three concept categories, and three erasure strategies revealing that all tested methods leave measurable residual capacity whose robustness correlates with intervention depth; and (3) the identification of temporal re-emergence, a video-specific failure mode where suppressed concepts progressively resurface across frames, invisible to frame-level metrics. These findings suggest that current erasure methods achieve output-level suppression rather than representational removal. We release our protocol to support reproducible safety auditing. Our code is available at https://github.com/YiweiXie/PRObingBasedEvaluation.
Figures & tables
Fig. 1 : Overview of the PROBE diagnostic framework.
Origin
NegPrompt
SAFREE
T2VUnlearning
Object Class
T1
T5
CL
T1
T5
CL
T1
T5
CL
T1
T5
CL
Cassette player
9.41
39.71
.175
1.76 / 2.65
21.76 / 25.00
.120 / .139
0.59 / 2.65
18.82 / 13.53
.104 / .112
0.00 / 1.18
8.82 / 19.41
.092 / .102
Chain saw
42.35
63.82
.146
18.24 / 22.06
41.76 / 44.41
.118 / .119
5.59 / 6.76
14.71 / 17.35
.110 / .117
0.00 / 0.00
3.24 / 1.47
.091 / .087
Church
59.70
77.65
.160
24.12 / 30.00
47.65 / 60.00
.147 / .152
3.53 / 6.47
15.88 / 37.35
.123 / .144
19.12 / 21.76
51.18 / 45.00
.141 / .141
Eng. springer
10.59
16.76
.141
11.47 / 11.76
24.12 / 32.35
.134 / .142
0.00 / 3.82
6.18 / 8.82
.088 / .097
5.59 / 6.67
7.94 / 11.18
.128 / .133
French horn
80.29
93.24
.188
24.41 / 29.71
50.59 / 66.18
.121 / .208
7.35 / 10.88
26.18 / 22.06
.171 / .177
1.47 / 3.53
10.00 / 15.59
.148 / .158
TABLE I : Object erasure results on CogVideoX-2B across three metrics (Erased / After PROBE). T1: Top-1 accuracy (%), T5: Top-5 accuracy (%), CL: CLIP similarity. The highest post-PROBE recovery per row per metric is in bold .
Fig. 2 : Temporal reactivation curve of parachute across different erasure strategies on CogX-2B.
Metrics
Origin
NegPrompt
SAFREE
T2VUnlearning
Mean Var
0.1834 0.0011
0.0819 / 0.1080 0.0009 / 0.0012
0.0514 / 0.0654 0.0007 / 0.0011
0.0137 / 0.0183 0.0002 / 0.0003
TABLE II : Temporal reactivation statistics for object erasure on CogX-2B (Erased / After PROBE).
Model
Origin
NegPrompt
SAFREE
T2VUnlearning
CogX-2B
56.14
42.82 / 56.16
35.16 / 56.16
19.63 / 28.90
CogX-5B
56.08
47.08 / 61.76
38.48 / 58.32
11.12 / 12.40
Wan2.2-5B
67.28
56.12 / 65.28
45.76 / 51.68
18.64 / 31.28
TABLE III : Nudity rate (%) across three T2V models (Erased / After PROBE).
Fig. 3 : Temporal reactivation curve of nudity across different erasure strategies on CogX-2B.
Exposed Class
Origin
NegPrompt
SAFREE
T2VUnlearning
CogVideoX-2B
Female genitalia
213
91 / 111
82 / 130
48 / 48
Male genitalia
13
14 / 17
86 / 33
56 / 69
Female breast
1003
631 / 910
397 / 536
377 / 615
Male breast
75
70 / 43
78 / 51
8 / 25
Buttocks
650
449 / 925
470 / 700
168 / 223
TABLE IV : Class-level NudeNet detections on CogVideoX-2B and CogVideoX-5B (Erased / After PROBE).
Model
Origin
NegPrompt
SAFREE
T2VUnlearning
CogX-2B
0.1597
0.1421 / 0.1522
0.1417 / 0.1516
0.1309 / 0.1385
CogX-5B
0.1422
0.1334 / 0.1444
0.1285 / 0.1483
0.1039 / 0.1036
Wan2.2-5B
0.1333
0.1286 / 0.1395
0.1159 / 0.1303
0.1029 / 0.1188
Average
0.1451
0.1347 / 0.1454
0.1287 / 0.1434
0.1126 / 0.1203
TABLE V : CLIP-based similarity for NSFW concepts across three T2V models (Erased / After PROBE).
Model
Metrics
Origin
NegPrompt
SAFREE
T2VUnlearning
CogX-2B
Mean Var
0.3201 0.0003
0.2566 / 0.3146 0.0001 / 0.0002
0.2163 / 0.2497 0.0001 / 0.0001
0.1212 / 0.1851 0.0001 / 0.0000
CogX-5B
Mean Var
0.3387 0.0000
0.2891 / 0.3550 0.0001 / 0.0001
0.2464 / 0.3625 0.0001 / 0.0001
0.0824 / 0.0994 0.0000 / 0.0000
Wan-5B
Mean Var
0.4437 0.0001
0.3660 / 0.4171 0.0001 / 0.0001
0.3003 / 0.3370 0.0000 / 0.0000
0.1364 / 0.2059 0.0000 / 0.0000
TABLE VI : Temporal reactivation statistics for nudity erasure (Erased / After PROBE).
Metrics
Origin
NegPrompt
SAFREE
T2VUnlearning
CogX-2B
Avg. Score Std_Dev
1.150 0.484
1.410 / 1.490 0.367 / 0.433
1.040 / 0.950 0.629 / 0.331
0.550 / 0.300 0.448 / 0.343
Fleiss’ κ
0.2577
CogX-5B
Avg. Score Std_Dev
1.440 0.613
1.420 / 1.310 0.410 / 0.507
1.070 / 1.570 0.579 / 0.313
0.490 / 0.540 0.595 / 0.642
Fleiss’ κ
0.3457
Wan-5B
Avg. Score Std_Dev
1.250 0.803
1.520 / 1.560 0.555 / 0.433
0.700 / 1.270 0.751 / 0.598
0.770 / 1.060 0.442 / 0.409
Fleiss’ κ
0.3362
TABLE VII : Human validation of nudity erasure (Erased / After PROBE).
Model
Identity
Origin
Erased
After PROBE
CogX-2B
Obama
0.4653
0.2684
0.2834
Trump
0.6503
0.2621
0.2966
CogX-5B
Obama
0.6548
0.0600
0.0945
Trump
0.6233
0.1232
− 0.0066
TABLE VIII : ID-similarity for celebrity identity recovery on CogVideoX with T2VUnlearning.
Fig. 4 : Temporal reactivation curve of identity on CogX-2B.
Model
Metrics
Origin
T2VUnlearning
CogX-2B
Mean Var
0.5578 0.0008
0.2653 / 0.2900 0.0004 / 0.0001
CogX-5B
Mean Var
0.6391 0.0004
0.0916 / 0.0440 0.0004 / 0.0002
TABLE IX : Temporal reactivation statistics for identity erasure (Erased / After PROBE).
Model
T2VUnlearning
P4D-K
PROBE
CogX-2B
19.63
20.45
28.90
TABLE X : Comparison of PROBE with P4D-K on CogX-2B under T2VUnlearning. Nudity rate (%).
Model
Origin
NegPrompt
SAFREE
T2VUnlearning
CogX-2B
36.45
22.14 / 26.53
17.44 / 32.27
11.37 / 13.95
TABLE XI : Cross-dataset evaluation on Ring-A-Bell. Nudity rate (%) on CogX-2B (Erased / After PROBE).
Transfer Direction
Erased Baseline
After Transfer
CogX-2B → Wan2.2-5B
18.64
15.80
CogX-5B → Wan2.2-5B
18.64
17.24
CogX-2B → CogX-5B
11.12
12.92
Wan2.2-5B → CogX-5B
11.12
11.00
TABLE XII : Cross-model transferability. Nudity rate (%) under T2VUnlearning.
Transfer Direction
Erased Baseline
After Transfer
T2VUnlearning → NegPrompt
42.82
60.02
T2VUnlearning → SAFREE
35.16
41.63
TABLE XIII : Cross-method transferability on CogX-2B. Nudity rate (%).
Metric
Origin
NegPrompt
SAFREE
T2VUnlearning
Imaging Quality
0.467
0.477 / 0.519
0.446 / 0.460
0.380 / 0.396
Aesthetic Quality
0.460
0.481 / 0.490
0.459 / 0.464
0.424 / 0.433
TABLE XIV : Visual quality on CogX-2B (Erased / After PROBE). Higher is better.
Model (Concept)
Erased
w/o Lalign
w/ Lalign
CogX-2B (Object)
7.44
8.56
10.32
CogX-2B (Nudity)
19.63
22.16
28.90
CogX-5B (Nudity)
11.12
11.36
12.40
Wan2.2-5B (Nudity)
18.64
27.92
31.28
TABLE XV : Ablation studies on T2VUnlearning. (Left) Reactivation rates (%) w/ and w/o the alignment loss Lalign . (Right) Nudity reactivation rates (%) under varying penalty weights λ .
Erased
10
50
100
200
Reference size
19.63
25.60
25.92
28.90
23.47
Erased
3 tokens
5 tokens
10 tokens
Token count
19.63
27.18
28.90
25.20
TABLE XVI : Sensitivity to reference data size and pseudo-token count on CogX-2B under T2VUnlearning (nudity rate, %).
CogX-2B_Object erasure
Metrics
Origin
NegPrompt
SAFREE
T2VUnlearning
Avg. Score
1.340
1.215 / 0.835
0.660 / 0.790
0.695 / 0.515
Std_Dev
0.593
0.593 / 0.627
0.620 / 0.548
0.670 / 0.545
Fleiss’ κ
0.3391
Identity erasure
Metrics
Origin
T2VUnlearning
TABLE XVII : Human validation of object and identity erasure on CogX-2B (Erased / After PROBE).
Fig. 5 : Attention heatmaps across different erasure methods. Rows: original model, erased, PROBE. Columns: different object categories. PROBE recovers coherent spatial attention patterns even after weight-space erasure.
Fig. 6 : Temporal reactivation curve of nudity across different erasure strategies on CogX-5B.
Fig. 7 : Temporal reactivation curve of nudity across different erasure strategies on Wan2.2-5B.
Fig. 8 : Temporal reactivation curve of identity erasure on CogX-5B.
Setting
Origin
λ=0
λ=1
T2VUnlearning
78.61 → 16.67
27.78
20.45
TABLE XVIII : Nudity rate (%) on HunyuanVideo under T2VUnlearning and PROBE with different λ values.
Text-to-video (T2V) diffusion models can reproduce copyrighted, violent, or explicit content, which motivates concept erasure: removing designated concepts from a pretrained model while preserving its behavior on everything else. Existing T2V erasure methods leave two problems open. Their frame-agnostic suppression can leave isolated frames in which an erased concept resurfaces, a frame-reactivation gap that clip-level averages obscure; and they are usually evaluated with one target concept or category at a time. We propose Frame-Aware Diffusion Erasure (FADE), a multi-concept video unlearning framework. FADE first applies a joint closed-form key/value edit that suppresses all target concepts, then trains per-concept frame-aware low-rank adapters whose strength is gated by the frame index and the denoising timestep to remove residual per-frame leakage. Each adapter is trained with the other targets' prompts as hard negatives, which keeps the concept-specific components of different adapters well separated, and a similarity-based soft router combines the adapters according to the prompt. With 16 concepts (objects, artistic styles, and nudity) erased from a single Wan2.1-T2V-1.3B backbone, FADE reduces the residual accuracy on the object benchmark to 4.9%, against 15.5% for the strongest of eight baselines, while keeping the VBench average within 0.9% of the unedited model. The ranking is unchanged under a VLM judge and a blinded human study, and the advantage over the strongest baseline carries over to prompts that combine several erased concepts, to 30 simultaneously erased celebrity identities, and to Wan2.1-T2V-14B, CogVideoX-2B, and HunyuanVideo-1.5.
Recent advances in text-to-video (T2V) diffusion models have demonstrated remarkable generative capabilities, yet their reliance on loosely curated training data raises pressing safety and copyright concerns. Concept erasure offers a principled remedy by removing unwanted semantics from pretrained models while preserving remaining concepts. However, existing approaches typically operate at a coarse granularity misaligned with the fine-grained, distributed nature of concept representations, leading to incomplete removal or degraded generation quality. We argue that surgical erasure fundamentally requires intervention at the level of monosemantic features, where each unit encodes a single interpretable concept. To this end, we propose EraseSAE, a novel framework that leverages sparse autoencoders to achieve surgical concept erasure in DiT-based T2V diffusion models via a principled decompose-attribute-erase pipeline. We first introduce the Partitioned Convolutional Sparse Autoencoder, which decomposes dense spatiotemporal activations into disentangled, interpretable sparse features while preserving spatiotemporal coherence. A contrastive attribution mechanism then contrasts activations from paired prompts to isolate concept-specific feature kernels. At inference, timestep-resolved spatiotemporal masks derived from the identified kernels confine erasure to regions where the target concept is active, leaving unrelated content intact. Extensive experiments across diverse diffusion models and concept erasure tasks demonstrate that EraseSAE achieves precise and robust concept removal with minimal quality degradation, substantially outperforming state-of-the-art methods. The code is available at https://github.com/HiDream-ai/EraseSAE.
Xinghao Wang, Dong Li, Wei Yu +5
University of Science and Technology of China, Hefei, China · HiDream.ai Inc. · Anhui Province Key Laboratory of Digital Security, China
Text-to-video (T2V) diffusion models can generate realistic depictions of actions such as kicking, stabbing, and shooting, raising safety concerns that motivate targeted concept erasure. Although concept erasure has been extensively studied for static concepts in text-to-image and T2V models, erasing motion concepts remains largely unexplored. We present a systematic study of motion concept erasure in video Diffusion Transformers (DiTs). Through causal interventions, we show that text-conditioning attention carries concept-specific motion information and supports selective intervention, whereas perturbing temporal positional encoding suppresses both target and non-target dynamics. We further find that directly adapting ESD, a representative weight-level image erasure method, to a video DiT yields modest and uneven motion suppression: reducing its erasure training loss does not by itself remove the concept signal from the difference between the conditional and unconditional predictions, which classifier-free guidance (CFG) then scales at every denoising step. From these findings, we derive three requirements for motion concept erasure: concept specificity, spatial selectivity, and temporal naturalness. Each determines one component of MUTE (Motion concept Unlearning in Text-to-video gEneration): at each denoising step, MUTE extracts a concept direction through token neutralization, derives a spatial gate from the direction's intrinsic structure, and subtracts the resulting correction from the velocity output before CFG is applied. MUTE is training-free and requires no weight modification. Experiments on 20 motion concepts show that MUTE outperforms representative prompt-level, weight-level, and inference-time baselines on Wan2.1-T2V, and the same formulation transfers to CogVideoX, supporting its applicability across distinct T2V attention architectures.
Ping Liu, Chi Zhang
Computer Science and Engineering University of Nevada, Reno Reno, NV, USA · Department of Mathematics National University of Singapore Singapore