VisualErase: Dual-Branch Visual Trajectory Redirection for Robust Concept Erasure in Text-to-Image Diffusion Models

Qianlong Xiang; Miao Zhang; Kun Wang; Yupeng Hu; Junhui Hou; Liqiang Nie

cs.CVOct 4, 2026

VisualErase: Dual-Branch Visual Trajectory Redirection for Robust Concept Erasure in Text-to-Image Diffusion Models

Authors: Qianlong Xiang, Miao Zhang, Kun Wang, Yupeng Hu, Junhui Hou, Liqiang Nie

Organizations: Harbin Institute of Technology (Shenzhen) · City University of Hong Kong · Shenzhen Loop Area Institute · National University of Singapore · Shandong University

Abstract

Concept erasure is essential for the safe deployment of text-to-image diffusion models, as they may reproduce harmful, copyrighted, or privacy-sensitive content learned from unconstrained large-scale data. Existing methods typically erase unwanted concepts while preserving general generation capability by redirecting target-related text-to-image mappings. However, recent studies show that erased models may still retain visual generative trajectories of target concepts, leaving them vulnerable to adversarial recovery attacks and revealing a fundamental gap between redirecting text-to-image mappings and truly removing visual knowledge. To bridge this gap, we propose VisualErase, a new paradigm that redirects concept-bearing visual generative trajectories toward explicitly defined concept-removed outcomes. To enable this redirection, we use structure-preserving image editing to construct content-aligned, concept-removed counterparts for source images, providing explicit visual endpoints that retain non-target content. We then derive a denoising target from each source-to-counterpart pair and use a dual-branch redirection loss to align both text-conditioned and unconditional predictions with this target, since conditional supervision alone does not explicitly constrain generation without textual guidance. To mitigate the adverse effects of concept erasure on non-target generation, we jointly optimize the redirection loss with a counterpart retention loss that matches denoising predictions from the frozen pretrained model. Across style, celebrity, and nudity erasure, VisualErase limits the maximum attack success rate over seven attacks to 0%, 8%, and 0.1%, respectively, while retaining general generation quality. These results highlight the importance of visual trajectory redirection for robust concept erasure beyond text-to-image mappings alone.

Figures & tables

Figure 1: Conceptual comparison of text-to-image redirection and visual trajectory redirection. (a) Text-to-image redirection steers generation under a target text condition toward an alternative concept, but does not necessarily remove the underlying ability to generate the target concept. (b) VisualErase instead supervises denoising from source visual states at different timesteps toward a content-aligned, concept-removed counterpart that defines the desired visual endpoint. Red arrows indicate the respective redirection objectives.

Figure 2: Overview of VisualErase . (a) Paired Data Construction creates a content-aligned counterpart in which the target concept is absent. (b) For visual trajectory erasure, Target Derivation converts the pair into a denoising target, which Dual-Branch Redirection applies under both the source and empty text conditions. Counterpart Retention is described in the text.

Erasure method	ACC $\downarrow$	Attack Success Rate (%) $\downarrow$							Generation utility
Erasure method	ACC $\downarrow$	PEZ	MMA	RAB	P4D	UDA	CCE	TINA+	FID $\downarrow$	CLIP $\uparrow$
SD1.4	76.0	40.0	40.0	90.0	94.0	96.0	60.0	76.0	14.0	26.6
ESD	2.0	2.0	0.0	6.0	30.0	32.0	8.0	60.0	14.5	25.9
FMN	10.0	0.0	2.0	6.0	54.0	56.0	18.0	70.0	13.8	26.5
AC	12.0	0.0	6.0	14.0	68.0	77.0	14.0	72.0	14.0	26.5
MACE	4.0	0.0	0.0	4.0	42.0	56.0	26.0	72.0	12.4	23.5

Table 1: Van Gogh style erasure.

Figure 3: Qualitative comparison across style, celebrity, and nudity erasure under standard generation and TINA+. Each setting compares SD1.4, ESD, STEREO, and VisualErase on the same evaluation cases. Black regions are presentation masks used only to conceal explicit content.

Erasure method	ACC $\downarrow$	Attack Success Rate (%) $\downarrow$							Generation utility
Erasure method	ACC $\downarrow$	PEZ	MMA	RAB	P4D	UDA	CCE	TINA+	FID $\downarrow$	CLIP $\uparrow$
SD1.4	35.6	31.4	34.8	70.5	40.7	39.8	3.5	42.4	14.0	26.6
ESD	2.5	0.8	0.7	20.0	3.5	5.6	46.5	39.0	13.6	25.6
FMN	33.9	30.5	31.1	69.5	38.0	38.7	15.5	44.9	13.8	26.3
UCE	0.0	4.2	6.9	8.4	7.0	7.7	14.8	41.5	14.3	26.4
MACE	0.0	1.7	0.1	0.0	5.6	7.7	17.6	43.2	12.8	24.2

Table 3: Nudity erasure under the VLM-based semantic evaluation protocol.

Variant	TINA+ ASR $\downarrow$	FID $\downarrow$	CLIP $\uparrow$
Cond.	66.0	16.0	26.8
Uncond.	2.0	25.3	26.5
Cond. + Uncond.	0.0	22.0	26.5
Cond. + Uncond. + Retent. ( VisualErase )	0.0	14.7	26.5

Table 4: Objective ablation on Van Gogh style erasure.

Trainable scope	Updated (M)	UDA ASR $\downarrow$	TINA+ ASR $\downarrow$	FID $\downarrow$	CLIP $\uparrow$
Cross-KV	19.2	2.0	46.0	21.1	26.1
Cross-Attn	44.0	0.0	2.0	17.4	26.7
Self-Attn	49.6	6.0	0.0	14.2	26.6
All-Attn ( VisualErase )	93.5	0.0	0.0	14.7	26.5
Non-Cross	815.4	2.0	0.0	14.5	26.5
All Linear/Conv	859.3	0.0	0.0	14.4	26.5

Table 5: Trainable U-Net parameter scope on Van Gogh style erasure under the complete VisualErase objective. Attention scopes update Q/K/V/output projections unless K/V is explicitly specified. Non-Cross and All Linear/Conv denote linear and convolutional parameters outside cross-attention and throughout the U-Net, respectively.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Comparison of paired-data construction strategies. Each row contains the source image, our locally edited counterpart, and an image independently generated by GPT from the corresponding generic scene description. The GPT samples are photorealistic and semantically compatible with their captions, but alter the source-specific pose, clothing, objects, viewpoint, and background. Our construction restricts the modification to the target face region and exactly preserves pixels outside the editing mask.

Figure 6: Representative NudeNet false positives without recognizable explicit anatomy.

Erasure method	ACC $\downarrow$	Attack Success Rate (%) $\downarrow$							Generation utility
Erasure method	ACC $\downarrow$	PEZ	MMA	RAB	P4D	UDA	CCE	TINA+	FID $\downarrow$	CLIP $\uparrow$
SD1.4	94.1	66.1	70.9	96.8	100.0	100.0	32.4	100.0	14.0	26.6
ESD	21.2	11.9	13.1	50.5	69.0	76.1	74.7	86.4	13.6	25.6
FMN	88.1	62.7	67.0	97.9	97.9	97.9	54.9	100.0	13.8	26.3
UCE	24.6	25.4	32.6	29.5	76.1	78.9	49.3	97.5	14.3	26.4
MACE	10.2	8.5	6.0	6.3	75.4	81.7	50.0	93.2	12.8	24.2

Appendix

Table 6: Classifier-only NudeNet evaluation for Nudity erasure at a threshold of 0.45.

Trainable scope	Active	Updated (M)	ACC $\downarrow$	TINA+ $\downarrow$	FID $\downarrow$	CLIP $\uparrow$
All-Attn	100%	93.5	0.0	6.8	15.5	26.8
Non-Cross	100%	815.4	0.0	0.8	16.7	26.3
All Linear/Conv	100%	859.3	0.0	0.8	16.3	26.4
SG Non-Cross	50%	407.7	0.0	0.0	16.7	26.4
SG Non-Cross	25%	203.9	0.8	0.8	16.0	26.5
SG Non-Cross ( VisualErase )	10%	81.5	0.0	0.0	15.5	26.6

Appendix

Table 7: Trainable scope and saliency ratio for Nudity erasure. All rows use natural clothed counterparts and 5,000 updates. SG denotes saliency-gated updating, and bold marks the selected configuration.

Method	Time (min)	Relative time
STEREO	30.0	2.12 $\times$
VisualErase	14.2	1.00 $\times$

Appendix

Table 8: Wall-clock training time for Van Gogh erasure on one H100.

Figure 7: VisualErase on Tench object erasure under standard target prompts and TINA+ attacks. The generated images do not contain a recognizable Tench.

Figure 8: Tench classifier predictions. All examples are predicted as Tench. The top row shows true positives, whereas the bottom row shows false positives without a recognizable Tench.

Explore similar work

Jul 6, 2026cs.CV

Erasing Without Collateral Damage: Precise Concept Removal in Diffusion Models

Training-free concept erasure is an attractive mechanism for controlling text-to-image diffusion models, but precise erasure often comes at the cost of damaging semantically related non-target concepts. Existing value-space methods remove the component of each cross-attention value along the target concept direction, implicitly treating target identity and shared visual structure as the same signal. We argue that this is the source of much of the collateral damage in prior preservation. We introduce CARE, a closed-form concept erasure operator that replaces the raw target direction with a kept-subspace-aware direction computed from a small bank of retained concept anchors. The resulting edit is applied directly in cross-attention value space, requires no model fine-tuning, and adds only a negligible offline computation. A single shrinkage parameter controls the erase-preserve trade-off. We further show that the operator admits a minimum-disturbance interpretation and, in its projection form, leaves the kept subspace invariant. Experiments under the standard concept-erasure protocol show that our method preserves non-target concepts more faithfully while maintaining competitive erasure across instance, style, and celebrity concepts. Code: https://github.com/parthupman/care

Parth Upman, Nishita Jain, Shreyank N Gowda

School of Computer Science, University of Nottingham, Nottingham, NG8 1BB, UK · Department of Computing, Imperial College London, London, SW7 2AZ, UK

Concept Erasure Text-To-Image Diffusion Models Diffusion Models Cross-Attention Semantic Anchor Structural Damage+1

Ask PDF

Oct 1, 2026cs.CV

Continual Concept Erasure in Diffusion Models by Suppressing Cross-Edit Interference

Concept erasure removes copyright-protected, privacy-sensitive, or otherwise undesirable concepts from pretrained text-to-image diffusion models to support content governance and compliance. As erasure requests arrive over time, models must remove new targets without undoing prior erasures. Existing methods do not constrain interference across edits: residual perturbations outside the retain set interact and accumulate, degrading unrelated generations and sometimes collapsing previously erased targets into noise. We propose CEASE (Continual Erasure via Adaptive Subspace Editing), a training-free method that imposes two subspace constraints on a closed-form solver. CEASE adds the token representation of the shared replacement to the solver's invariance matrix and, when interference is detected, projects the current update onto the orthogonal complement of dominant output directions extracted from cumulative past updates. A closed-form decomposition attributes the accumulated interference to repeated activation of the shared replacement and overlap between successive update directions, showing that the two constraints suppress these respective sources. Across continual erasure of celebrities, artistic styles, and instances, CEASE achieves the most consistent erase-preserve trade-off, while existing methods either degrade general generation or insufficiently erase targets.

Yongliang Wu, Haori Lu, Jinqi Luo +3

University of Illinois Urbana-Champaign · University of Pennsylvania · National University of Singapore

Concept Erasure Text-To-Image Diffusion Models Diffusion Models Content Moderation Interference Subspace

Ask PDF

Sep 14, 2026cs.CV

GRACE: Adaptive Concept Erasure with Geometry-Guided Retention in Diffusion Models

Text-to-image (T2I) diffusion models inevitably internalize sensitive or non-compliant concepts from large-scale pretraining data, necessitating post-hoc concept erasure. However, existing erasure methods often lack explicit constraints on parameter updates, leading to over-intervention and unintended semantic drift. In addition, many methods rely on manually crafted counterfactual supervision, such as surrogate prompts, which incurs substantial data construction costs that limit scalability to new concepts. To address these limitations, we propose GRACE, a structured concept erasure framework designed to enable localized and selective intervention. Specifically, we introduce a semantically weighted sensitive subspace estimation to precisely lock intervention directions, and employ lightweight subspace-constrained adapters to prevent global semantic disturbance. To eliminate the dependency on manual prompt engineering, we design an automatically decoupled safe-anchor mechanism. To mitigate semantic drift induced by excessive intervention, we introduce an energy-driven dynamic gating mechanism that adaptively controls the timing and strength of intervention at inference. Extensive experiments demonstrate that our method achieves a superior balance between erasure effectiveness and generation fidelity. Compared with the average performance of five state-of-the-art (SOTA) concept erasure methods, our method improves the fine-grained NSFW reduction rate by

17.86\%

, while reducing the macro-averaged target CLIP Score and preservation-oriented Fr'echet Inception Distance (FID) by

4.75\%

and

50.58\%

, respectively, indicating stronger concept suppression with substantially improved preservation of the original model's generative utility.

Qinghui Gong, Yihuai Liang, Yuanlun Xie +3

School of Information Science and Technology, Southwest Jiaotong University, Chengdu 611756, China · School of Electronic Information and Electrical Engineering, Chengdu University, Chengdu, China · Key Laboratory of Intelligent Control and Optimization for Industrial Equipment of Ministry of Education, Dalian University of Technology, Dalian, China +1

Concept Erasure Text-To-Image Diffusion Models Diffusion Models Fréchet Inception Distance

Ask PDF