Concept Score Relearning: A Unified Cross-Architecture Attack on Concept Erasure

Hong Xi Tae; Jiaming Zhang; Wenwen He; Xuan Wang; Wei Yang Bryan Lim

cs.CVSep 27, 2026

Concept Score Relearning: A Unified Cross-Architecture Attack on Concept Erasure

Authors: Hong Xi Tae, Jiaming Zhang, Wenwen He, Xuan Wang, Wei Yang Bryan Lim

Organizations: College of Computing and Data Science, Nanyang Technological University, Singapore

Abstract

Concept erasure aims to suppress undesirable knowledge in text-to-image generative models. However, existing robustness evaluations typically rely on relearning attacks tailored to specific model architectures. We study concept reactivation across two substantially different generative paradigms: noise-prediction U-Nets and flow-matching Transformers. We introduce \textbf{Concept Score Relearning (CSR)}, a unified parameter-level framework that reactivates erased concepts by optimizing each model within its native prediction space. CSR requires no external target-concept image dataset and applies the same concept-directed objective to both U-Net-based Stable Diffusion and Transformer-based FLUX. Experiments across diverse concepts and multiple erasure methods demonstrate consistent concept reactivation across both architectures, highlighting the cross-architecture applicability of CSR and the persistent recoverability of apparently erased concepts. For strict nudity, CSR reaches average ASRs of 50.47% on FLUX and 40.29% on Stable Diffusion, consistently ranking first across all evaluated safety settings.

Figures & tables

Figure 1: Visual result of CSR. Top row: ESD-erased models. Bottom row: after CSR. Across nudity, artistic style, abstraction, relationship, and entity concepts, CSR restores recognizable erased content on both FLUX.1-dev and SD-1.4. Yellow-framed insets show outputs from the corresponding unerased models. Each column shares the same prompt; concept words are teal, and explicit nudity is masked with blue bars.

Figure 2: Overview of Concept Score Relearning (CSR). CSR self-generates concept anchors from the frozen unerased reference model and reactivates erased concepts using the conditional–unconditional prediction difference in each architecture’s native prediction space. The same objective is applied to SD-1.4 in noise-prediction space and FLUX.1-dev in velocity-prediction space.

Panel of Figure 3 — Figure 3: Visual result of CSR across safety-related and artistic-style concepts. (a) NSFW: nudity and violence examples on FLUX.1-dev and SD-1.4 across multiple erasure methods, comparing the erased model without attack, Ring-A-Bell, and CSR. (b) Artistic styles: Vincent van Gogh and Pablo Picasso examples on both architectures under ESD, EA, and CP, comparing the erased model, ReFLUX, and CSR. Yellow-framed insets show the corresponding generations from the original unerased models, while blue bars mask explicit regions for presentation. Across both safety and artistic-style concepts, CSR more consistently restores visually recognizable target characteristics across architectures and erasure methods.

Figure 4: Visual result on Entity, Abstraction, and Relationship. Results are shown for (a) FLUX.1-dev and (b) SD-1.4. Rows correspond to the erased model, ReFLUX, and CSR. Yellow-framed insets show outputs from the corresponding unerased models, while blue bars mask explicit regions. CSR restores recognizable target concepts across both architectures.

Method	React.	Align.	Preserv.	Quality
ReFLUX	4.52 $\pm$ 0.78	3.89 $\pm$ 1.29	3.91 $\pm$ 0.91	4.03 $\pm$ 0.44
Ours	4.62 $\pm$ 0.91	4.74 $\pm$ 0.22	4.68 $\pm$ 0.53	4.52 $\pm$ 0.76

Table 5: Blinded human evaluation. Mean

\pm

standard deviation of participant-level average ratings on a 1–5 Likert scale; higher is better. Participants evaluate ReFLUX and CSR on concept reactivation (React.), prompt alignment (Align.), irrelevant-content preservation (Preserv.), and image quality.

Panel of Figure 5 — Figure 5: Key ablations of CSR under ASR-Strict (%). (a) Effect of restoration strength $\eta$ , with the dashed line marking the default $\eta=3$ . (b) Effect of the number of self-generated concept anchors $N$ . (c) Effect of anchor source. Results use ESD-erased nudity with attack seed 0 on SD-1.4 and FLUX.1-dev. Full ablations are provided in Appendix A.7.4 .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description
$x$	Image in pixel space
$\mathcal{E},\mathcal{D}$	VAE encoder and decoder
$z_{0}$	Clean latent
$z_{t}$	Generic perturbed latent in CSR; noisy latent at discrete diffusion timestep $t$ for U-Net-based Stable Diffusion
$t$	Generic timestep/noise-level notation in CSR; discrete diffusion timestep for U-Net-based Stable Diffusion
$\bar{\alpha}_{t}$	Cumulative diffusion noise-schedule coefficient for U-Net-based Stable Diffusion

Appendix

Table 6: Notation used throughout the paper. Principal symbols for CSR and its architecture-specific instantiations on U-Net-based Stable Diffusion and Transformer-based FLUX.

Aspect	U-Net-based Stable Diffusion	Transformer-based FLUX
Backbone	Convolutional U-Net	Multimodal Transformer
Generative formulation	Latent diffusion	Flow matching
Forward perturbation	Variance-scheduled Gaussian noising	Linear data–noise interpolation
Time/noise parameter	Discrete timestep $t$	Continuous noise level $s$
Native prediction	Predicted noise $\epsilon_{\theta}$	Predicted velocity field $v_{\theta}$
Text conditioning	U-Net cross-attention	Multimodal Transformer attention

Appendix

Table 7: Architectural comparison of U-Net-based Stable Diffusion and Transformer-based FLUX. The two model families differ in architecture, perturbation process, and native prediction parameterization, while CSR applies the same conditional–unconditional target construction within each model’s native prediction space.

Configuration	SD-1.4	FLUX.1-dev
Native prediction	Predicted noise	Predicted velocity field
Anchor images	40	40
Image sampling steps	25	28
Sampling guidance	7.5	3.5
Reference targets	Online	Cached
Cached tuples	–	$40\times 12=480$

Appendix

Table 8: Architecture-specific CSR configurations. The same concept-directed target and relearning objective are used for both architectures, while optimization and reference-evaluation settings differ according to the underlying model.

Figure 6: Qualitative comparison of ASR-Strict and ASR-Permissive detections. Top: examples detected by ASR-Strict and therefore also by ASR-Permissive through NudeNet classes corresponding to explicit nudity. Bottom: examples detected only by ASR-Permissive through permissive-only classes such as exposed belly, armpits, male chest, or feet. Blue bars mask explicit regions for presentation, while orange boxes indicate regions associated with permissive-only detections.

Victim	ASR-Strict	ASR-Permissive
EAP	53.68 (2.73 $\times$ )	71.93 (1.39 $\times$ )
MACE	58.25 (2.96 $\times$ )	75.09 (1.46 $\times$ )
ESD	33.33 (1.70 $\times$ )	48.42 (0.94 $\times$ )
CP	51.58 (2.62 $\times$ )	70.53 (1.37 $\times$ )

Appendix

Table 9: Comparison of ASR-Strict and ASR-Permissive on FLUX.1-dev. Values in parentheses denote the ratio to the corresponding unerased-model ASR (

19.65\%

for ASR-Strict and

51.58\%

for ASR-Permissive). The higher unerased detection rate under ASR-Permissive compresses the normalized reactivation ratio relative to ASR-Strict.

Entity	Abstraction	Relationship
A photo of fruit	A scene featuring explosion	A shake hand B
A photo of ball	A scene featuring green bag	A kiss B
A photo of car	A scene featuring yellow bag	A hug B
A photo of airplane	A scene featuring time	A in B
A photo of tower	A scene featuring two cats	A on B
A photo of building	A scene featuring three cats	A back to back B

Appendix

Table 10: Concept inventory for the miscellaneousness evaluation. The benchmark contains ten Entity, ten Abstraction, and ten Relationship concepts using the original prompts released with ReFLUX ( Jiang et al., 2025 ) .

Architecture	Excluded Concept	Reference Detection Accuracy
SD-1.4	A scene featuring two cats	0%
	A in B	0%
	A on B	0%
	A hold B	10%
	A amidst B	10%
	A scene featuring explosion	30%

Appendix

Table 11: Concepts excluded by the reference-model ceiling filter. Filtering is performed independently for SD-1.4 and FLUX.1-dev. Values report the detection accuracy of the corresponding unerased reference model before concept erasure.

Setting	ASR-Strict	ASR-Permissive
Restoration strength $\eta$
$\eta=1$ (reference matching)	17.89	34.04
$\eta=2$	28.07	43.86
$\eta=3$	32.98	49.47
$\eta=4$	29.82	50.88
Number of anchors $N$

Appendix

Table 14: Full CSR ablation results on SD-1.4. ASR-Strict and ASR-Permissive (%) are reported for the ESD nudity victim. Yellow rows indicate the default settings used in the main experiments, while bold denotes the nominal best result within each ablation group. Each cell corresponds to a single seed-0 attack run.

Setting	ASR-Strict	ASR-Permissive
Restoration strength $\eta$
$\eta=1$ (reference matching)	10.88	31.23
$\eta=2$	13.33	33.33
$\eta=3$	38.25	51.93
$\eta=4$	30.53	43.51
Number of anchors $N$

Appendix

Table 15: Full CSR ablation results on FLUX.1-dev. ASR-Strict and ASR-Permissive (%) are reported for the ESD nudity victim. Yellow rows indicate the default settings used in the main experiments, while bold denotes the nominal best result within each ablation group. Each cell corresponds to a single seed-0 attack run.

Figure 7: Blinded human evaluation of concept reactivation. Bars show mean participant-level average ratings on a 1–5 Likert scale, with error bars indicating standard deviations. ReFLUX and CSR are evaluated on concept reactivation, prompt alignment, irrelevant-content preservation, and image quality. Higher is better.

Explore similar work

FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Models
May 19, 2026Yi Sun, Zhiqi Zhang, Xinhao Zhong +5Concept Erasure Generative Flow Networks
Harbin Institute of Technology, Shenzhen · Jilin University · Tsinghua Shenzhen International Graduate School, Tsinghua University +2
You Can't Have It Both Ways: Concept Entanglement Limits Diffusion Model Unlearning
Sep 28, 2026Yian Wang, Ali Ebrahimpour-Boroojeny, Hari Sundaram +1Text-To-Image Diffusion Models Concept Erasure
University of Illinois Urbana-Champaign Urbana, IL, USA
GRACE: Adaptive Concept Erasure with Geometry-Guided Retention in Diffusion Models
Sep 14, 2026Qinghui Gong, Yihuai Liang, Yuanlun Xie +3Concept Erasure Text-To-Image Diffusion Models
School of Information Science and Technology, Southwest Jiaotong University, Chengdu 611756, China · School of Electronic Information and Electrical Engineering, Chengdu University, Chengdu, China · Key Laboratory of Intelligent Control and Optimization for Industrial Equipment of Ministry of Education, Dalian University of Technology, Dalian, China +1

May 19, 2026, cs.CVJ/K move · Enter open · S save

FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Models

Yi Sun, Zhiqi Zhang, Xinhao Zhong, Yimin Zhou+4

Harbin Institute of Technology, Shenzhen · Jilin University · Tsinghua Shenzhen International Graduate School, Tsinghua University+2

Concept Erasure Generative Flow Networks Text-To-Image Diffusion Models Clover

Open paper AskPDF arXiv

Recent advances in flow matching models have significantly improved text-to-image generation quality, but also introduce growing safety risks due to the generation of harmful or undesirable content. Existing concept erasure methods are either inference-time interventions with limited effectiveness or rely on supervised fine-tuning (SFT), which requires precisely aligned data and struggles with scalability and multi-concept settings. In this paper, we propose \emph{FlowErase-RL}, the first GRPO-based framework for concept erasure in flow matching models. We reformulate concept erasure as a reward optimization problem and introduce a \textbf{dynamic dual-path reward mechanism} that jointly optimizes (i) a Concept Erasure (CE) reward to suppress target concepts and (ii) a Non-target Space (NS) reward to preserve generative fidelity. The two reward paths are adaptively balanced during training via a performance-driven switching strategy, enabling stable optimization without explicit supervision. Extensive experiments on nudity, object, and artistic style erasure demonstrate that our method achieves state-of-the-art erasure performance while maintaining strong image quality and semantic alignment. Moreover, it exhibits robust resistance to adversarial attacks and scales effectively to multi-concept scenarios. Our results establish a new paradigm for safe and controllable generation in flow matching models.

Abstract

Explore similar work

FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Models

You Can't Have It Both Ways: Concept Entanglement Limits Diffusion Model Unlearning

GRACE: Adaptive Concept Erasure with Geometry-Guided Retention in Diffusion Models