TINA+: Probing Residual Visual Knowledge in Unlearned Diffusion Models via Diffusion-Consistent Text-Free Inversion
Authors: Qianlong Xiang, Miao Zhang, Kun Wang, Haoyu Zhang, Junhui Hou, Liqiang Nie
Organizations: School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen 518055, China · City University of Hong Kong, Hong Kong SAR, China · Shenzhen Loop Area Institute, Shenzhen 518045, China · Harbin Institute of Technology (Shenzhen), Shenzhen 518055, China · National University of Singapore, Singapore · Pengcheng Laboratory, Shenzhen 518000, China · Department of Computer Science, City University of Hong Kong, Hong Kong, China
Although text-to-image diffusion models exhibit remarkable generative power, concept erasure techniques are essential for preventing harmful content. Existing adversarial probes evaluate these methods by testing whether erased concepts can still be recovered. However, existing erasure and probe methods remain largely text-centric, focusing on whether the text-to-image mapping is severed while overlooking whether the corresponding visual knowledge remains. To investigate this question from a visual perspective, we leverage diffusion inversion to probe whether a generative trajectory can reconstruct visual instances of an erased concept. Under a null-text condition, standard inversion avoids the textual pathway but amplifies approximation errors, hindering faithful trajectory recovery. To address this challenge, we introduce TINA+, a diffusion-consistent Text-free INversion Attack equipped with optimization-based inversion. We also find that unconstrained diffusion inversion may discover spurious trajectories, even allowing a randomly initialized diffusion model to reconstruct the target concept. Such trajectories may falsely indicate residual visual knowledge. TINA+ therefore introduces Diffusion-Consistent Trajectory Regularization to suppress this failure mode. By penalizing trajectories that fall far below the expected marginal energy evolution of diffusion, TINA+ suppresses spurious inversion paths while preserving its ability to recover erased concepts. Experiments across twelve erasure methods, four concept-erasure tasks, and different model architectures demonstrate that TINA+ reliably probes residual visual knowledge through diffusion-consistent visual trajectories. These results provide stronger evidence that current methods often obscure concepts by severing text-image links rather than eliminating the underlying visual knowledge.
Figures & tables
Fig. 1: Conceptual overview of text-centric erasure vulnerabilities and our TINA+ attack. Concept Erasure usually severs the link between a specific text condition and the undesired concept. Previous Attacks remain text-centric, finding adversarial text condition to reactivate the concept. Our TINA+ bypasses the text pathway entirely. Using an empty text condition, it finds a noise that regenerates the concept through a diffusion-consistent trajectory, providing stronger evidence that residual visual knowledge persists in existing erased models.
Fig. 2: Framework overview of our proposed attack. Given target images containing the erased concept, we identify generative trajectories and obtain the corresponding seed latents. These seed latents are then used to generate new images with the same concept-erased model. If the generated images still contain the concept that should have been erased, the model is considered to retain the corresponding visual knowledge.
Fig. 3: Standard DDIM inversion is insufficient for reliable visual trajectory discovery. (a) Text-conditioned inversion is resisted by the concept-erased model, since the corresponding text-to-image mapping has been suppressed. (b) Null-text inversion avoids the textual defense but suffers from accumulated inversion errors, failing to faithfully recover the target concept even on the unerased model. (c) Null-text inversion can even produce visually recognizable target-like images on a randomly initialized diffusion model through an unnatural, diffusion-inconsistent trajectory.
Fig. 4: Overview of TINA+. Given a target image, we encode it into a latent z0 and perform text-free inversion under the null-text condition. TINA+ first refines each intermediate latent through fixed-point consistency to correct accumulated inversion errors. Furthermore, TINA+ applies diffusion-consistent trajectory regularization, including forward marginal initialization and marginal energy loss, to avoid spurious diffusion-inconsistent trajectories. The resulting seed latent zT∗ is then used for generation with the same concept-erased model.
Fig. 5: Trajectory energy diagnostics on Stable Diffusion v1-4 and a randomly initialized UNet. The top row shows the marginal energy score rteng , while the bottom row compares the observed energy transition γt with the marginal energy transition γtmarg=1−αˉt . Stable Diffusion follows a reasonable diffusion energy transition, whereas the Random UNet exhibits severe energy collapse, indicating a diffusion-inconsistent trajectory.
Method
Venue
Year
Unlearning Methods
ESD [ 32 ]
ICCV
2023
AC [ 33 ]
ICCV
2023
FMN [ 34 ]
CVPRW
2024
UCE [ 35 ]
WACV
2024
SH [ 48 ]
ECCV
2024
TABLE I: Summary of the unlearning and attack methods evaluated in this work, together with their venues and years.
ESD
FMN
UCE
MACE
RECE
AdvUn.
SalUn
STEREO
AVG.
PEZ
11.86
62.71
25.42
8.47
15.25
1.69
0.00
0.00
15.68
MMA
13.10
67.00
32.60
6.00
22.80
1.70
1.70
5.50
18.80
RAB
50.53
97.89
29.47
6.32
10.53
2.11
0.00
8.42
25.66
P4D
69.01
97.89
76.06
75.35
66.20
18.31
15.49
24.65
55.37
UDA
76.05
97.89
78.87
81.69
63.38
23.24
13.38
25.35
57.48
TINA+
86.44
100.00
97.46
93.22
93.22
92.37
73.73
97.46
91.74
TABLE II: Quantitative comparison of Attack Success Rates (ASR, in %) on the nudity erasure task. We evaluate our TINA+ against five baselines across eight prominent unlearning defenses. AdvUn. denotes AdvUnlearn. Bold indicates the best-performing attack.
Fig. 6: Qualitative comparison of PEZ, MMA, RAB, P4D, UDA, and TINA+ on the nudity erasure task. Each column corresponds to an unlearning defense. P4D, UDA, and TINA+ use the same I2P target, whereas PEZ, MMA, and RAB use representative cases from their native evaluation protocols. Sensitive content is redacted.
ESD
FMN
AC
MACE
SPM
RECE
AdvUn.
STEREO
AVG.
PEZ
2.0
0.0
0.0
0.0
5.0
0.0
0.0
0.0
0.9
MMA
0.0
2.0
6.0
0.0
14.0
14.0
0.0
0.0
4.5
RAB
6.0
6.0
14.0
4.0
48.0
20.0
0.0
0.0
12.3
P4D
30.0
54.0
68.0
42.0
78.0
62.0
0.0
0.0
41.8
UDA
32.0
56.0
77.0
56.0
88.0
64.0
2.0
0.0
46.9
TINA+
60.0
70.0
72.0
72.0
74.0
72.0
68.0
46.0
66.8
TABLE III: Comparison of Attack Success Rates (ASR, in %) on the artistic style erasure task. We report the performance of our TINA+ against five baselines across eight unlearning methods. AdvUn. denotes AdvUnlearn. Bold denotes the highest ASR.
FMN
SPM
ESD
EraseDiff
SalUn
SH
AdvUn.
STEREO
AVG.
Church
PEZ
32.00
34.00
2.00
2.00
12.00
0.00
0.00
0.00
10.25
MMA
32.00
26.00
6.00
10.00
8.00
0.00
0.00
2.00
10.50
RAB
58.00
68.00
14.00
28.00
4.00
0.00
0.00
0.00
21.50
P4D
94.00
94.00
58.00
42.00
62.00
0.00
2.00
8.00
45.00
UDA
98.00
94.00
66.00
50.00
68.00
4.00
8.00
10.00
49.75
TABLE IV: Attack Success Rates (ASR, in %) of TINA+ and baselines on object erasure across four concepts (Church, Garbage Truck, Parachute, and Tench). AdvUn. denotes AdvUnlearn. Bold indicates the best-performing attack.
Attack
Taylor Swift
Elon Musk
Adam Lambert
ESD
PEZ
2.00
0.00
4.00
MMA
2.00
0.00
2.00
RAB
12.00
6.00
4.00
P4D
98.00
100.00
90.00
UDA
98.00
100.00
92.00
TABLE V: Attack Success Rates (ASR, in %) on celebrity identity erasure under ESD and STEREO. Bold indicates the best result for each identity under the same defense.
Attack
Van Gogh
Nudity
Tench
ESD
CCE
8.00
74.65
40.00
TINA+
60.00
86.44
64.00
STEREO
CCE
4.00
16.90
2.00
TINA+
46.00
97.46
66.00
TABLE VI: Comparison between text inversion (CCE) and image inversion (TINA+) across style, nudity, and object erasure. Results are ASRs (in %). Bold indicates the better inversion strategy.
Fig. 7: Qualitative comparison on Van Gogh style and Tench object erasure under STEREO. Each row shows the VAE-reconstructed target, ordinary generation from the erased model, UDA, and TINA+. While the no-attack and UDA outputs deviate from the target concept or composition, TINA+ closely reconstructs the target-specific visual content.
Fig. 8: Qualitative identity recovery on ESD and STEREO. Each row uses one target image for Taylor Swift, Elon Musk, or Adam Lambert. The no-attack outputs weaken or alter the target identity, whereas TINA+ reconstructs identity-specific facial characteristics under both defenses. For each celebrity, we select an index for which TINA+ is recognized as the target identity by GCD on both ESD and STEREO.
Fig. 9: Roles of Forward Marginal Initialization (FMI) and marginal energy regularization in trajectory discovery on the Van Gogh benchmark. (a) On pretrained Stable Diffusion v1.4, FMI provides an initialization aligned with the expected marginal energy progression, and TINA+ follows the same plausible trajectory. (b) On the randomly initialized UNet negative control, FMI provides an aligned starting point but cannot prevent the subsequent energy collapse, whereas the full TINA+ objective substantially mitigates it. Curves report the mean over 50 target images.
Van Gogh
Taylor Swift
Method
LPIPS ↑
DreamSim ↑
DINO ↓
ASR ↓
LPIPS ↑
DreamSim ↑
DINO ↓
ASR ↓
DDIM
0.396
0.212
0.725
48.0
0.286
0.164
0.695
86.0
TINA
0.361
0.181
0.769
60.0
0.245
0.140
0.730
88.0
TINA w/FMI
0.482
0.270
0.633
28.0
0.484
0.256
0.588
60.0
TINA+
0.651
0.493
0.434
0.0
0.749
0.625
0.400
0.0
TABLE VII: False-positive suppression throughout the progression from DDIM inversion to TINA+ on the Van Gogh and Taylor Swift benchmarks. The randomly initialized UNet contains no learned target knowledge and therefore serves as a negative control: higher LPIPS and DreamSim, lower DINO, and lower ASR indicate stronger suppression of spurious target reconstruction.
Fig. 10: Qualitative false-positive reconstruction on the Taylor Swift benchmark using a randomly initialized UNet. Although the model contains no learned target knowledge, DDIM inversion and unconstrained TINA closely reconstruct the targets, while FMI alone only partially disrupts these spurious results. Marginal energy regularization substantially degrades the diffusion-inconsistent reconstructions, causing all three TINA+ outputs to fail the identity detector.
Fig. 11: t-SNE visualization of (a) the optimized initial noises zT∗ produced by TINA+ and (b) their corresponding deep ESD UNet activations, extracted from the last ResNet block of mid_block , for four erased concepts. Each concept contains 50 target images.
Fig. 12: Generalization of TINA+ to a DiT-based architecture ( PixArt-XL-2-512x512 ). Each column shows a target image, the ordinary output of the ESD-erased model, and the corresponding TINA+ reconstruction. ESD removes the Parachute concept under normal sampling, whereas TINA+ recovers diverse target-specific parachutes.
Although text-to-image diffusion models exhibit remarkable generative power, concept erasure techniques are essential for their safe deployment to prevent the creation of harmful content. This has fostered a dynamic interplay between the development of erasure defenses and the adversarial probes designed to bypass them, and this co-evolution has progressively enhanced the efficacy of erasure methods. However, this adversarial co-evolution has converged on a narrow, text-centric paradigm that equates erasure with severing the text-to-image mapping, ignoring that the underlying visual knowledge related to undesired concepts still persist. To substantiate this claim, we investigate from a visual perspective, leveraging DDIM inversion to probe whether a generative pathway for the erased concept can still be found. However, identifying such a visual generative pathway is challenging because standard text-guided DDIM inversion is actively resisted by text-centric defenses within the erased model. To address this, we introduce TINA, a novel Text-free INversion Attack, which enforces this visual-only probe by operating under a null-text condition, thereby avoiding existing text-centric defenses. Moreover, TINA integrates an optimization procedure to overcome the accumulating approximation errors that arise when standard inversion operates without its usual textual guidance. Our experiments demonstrate that TINA regenerates erased concepts from models treated with state-of-the-art unlearning. The success of TINA proves that current methods merely obscure concepts, highlighting an urgent need for paradigms that operate directly on internal visual knowledge.
Qianlong Xiang, Miao Zhang, Haoyu Zhang +3
Harbin Institute of Technology (Shenzhen) · City University of Hong Kong · Shenzhen Loop Area Institute +2
Concept erasure is essential for the safe deployment of text-to-image diffusion models, as they may reproduce harmful, copyrighted, or privacy-sensitive content learned from unconstrained large-scale data. Existing methods typically erase unwanted concepts while preserving general generation capability by redirecting target-related text-to-image mappings. However, recent studies show that erased models may still retain visual generative trajectories of target concepts, leaving them vulnerable to adversarial recovery attacks and revealing a fundamental gap between redirecting text-to-image mappings and truly removing visual knowledge. To bridge this gap, we propose VisualErase, a new paradigm that redirects concept-bearing visual generative trajectories toward explicitly defined concept-removed outcomes. To enable this redirection, we use structure-preserving image editing to construct content-aligned, concept-removed counterparts for source images, providing explicit visual endpoints that retain non-target content. We then derive a denoising target from each source-to-counterpart pair and use a dual-branch redirection loss to align both text-conditioned and unconditional predictions with this target, since conditional supervision alone does not explicitly constrain generation without textual guidance. To mitigate the adverse effects of concept erasure on non-target generation, we jointly optimize the redirection loss with a counterpart retention loss that matches denoising predictions from the frozen pretrained model. Across style, celebrity, and nudity erasure, VisualErase limits the maximum attack success rate over seven attacks to 0%, 8%, and 0.1%, respectively, while retaining general generation quality. These results highlight the importance of visual trajectory redirection for robust concept erasure beyond text-to-image mappings alone.
Qianlong Xiang, Miao Zhang, Kun Wang +3
Harbin Institute of Technology (Shenzhen) · City University of Hong Kong · Shenzhen Loop Area Institute +2
Large-scale text-to-image (T2I) diffusion models deliver remarkable visual fidelity but pose safety risks due to their capacity to reproduce undesirable content, such as copyrighted ones. Concept erasure has emerged as a mitigation strategy, yet existing approaches struggle to balance scalability, precision, and robustness, which restricts their applicability to erasing only a few hundred concepts. To address these limitations, we present Erasing Thousands of Concepts (ETC), a scalable framework capable of erasing thousands of concepts while preserving generation quality. Our method first models low-rank concept distributions via a Student's t-distribution Mixture Model (tMM). It enables pin-point erasure of target concepts via affine optimal transport while preserving others by anchoring the boundaries of target concept distributions without pre-defined anchor concepts. We then train a Mixture-of-Experts (MoE)-based module, termed MoEraser, which removes target embeddings while preserving the anchor embeddings. By injecting noise into the text embedding projector and fine-tuning MoEraser for recovery, our framework achieves robustness to white-box attack such as module removal. Extensive experiments on over 2,000 concepts across heterogeneous domains and diffusion models demerate state-of-the-art scalability and precision in large-scale concept erasure.
Hoigi Seo, Byung Hyun Lee, Jaehyun Cho +2
Dept. of Electrical and Computer Engineering, Seoul National University, Republic of Korea · IPAI, Seoul National University, Republic of Korea · INMC, Seoul National University, Republic of Korea