Beyond User-Side Defenses: Image Protection in Open-Weight Diffusion Models

Jihye Kang; Jiwon Park; Namhyuk Ahn

cs.CROct 12, 2026

Beyond User-Side Defenses: Image Protection in Open-Weight Diffusion Models

Authors: Jihye Kang, Jiwon Park, Namhyuk Ahn

Organizations: Department of Electrical and Computer Engineering, Inha University

Abstract

Model providers can support image protection, but open-weight deployment introduces practical constraints. We propose four requirements: protection effectiveness, clean fidelity, compatibility, and tamper resistance. We incorporate long-horizon attack simulations to strengthen resistance to protection removal. Experiments with a single protection tool show improved resistance to re-alignment at a cost to clean fidelity, motivating further research toward practical model-side protection.

Figures & tables

Figure 1: Maintaining image protection after open-weight release. A re-alignment attack removes the protective behavior of an aligned model (top), whereas tamper-resistance training helps preserve protection under the same attack (bottom). Each row shows edits of the same protected image as the attack progresses, with the corresponding clean-image edit on the right. $t$ : attack epochs.

Model	Protection $\uparrow$		Fidelity $\uparrow$ (dB)	Removal $\uparrow$ (epochs)
Model	No Attack	Attacked	Fidelity $\uparrow$ (dB)	Removal $\uparrow$ (epochs)
Original	0.225	N/A	N/A	N/A
Alignment	0.751	0.118	30.9	1
+ short	0.764	0.107	29.8	1
+ long	0.743	0.622	25.7	$\boldsymbol{>20}$

Table 1: Protection effectiveness, clean fidelity, and tamper resistance. Protection is measured by LPIPS before attack and after 20 epochs of encoder re-alignment. Fidelity is pre-attack reconstruction PSNR relative to the original autoencoder. Removal denotes the first attack epoch at which protection LPIPS falls to or below the unaligned reference of 0.225;

>20

means the threshold was not reached.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Protective Alignment
Quantity	Value
$\alpha_{i}$ , $\alpha_{l}$ , $\beta$ , $\gamma$	0.5, 0.5, 0.3, 2.0
$\tau$ , $w_{\mathrm{text}}$ , $w_{\mathrm{bg}}$	0.15, 3.0, 1.0
optimizer, epochs	AdamW, lr $10^{-5}$ , 80
trainable	encoder, quant. convs, decoder tail
Tamper-Resistance Training

Appendix

Table 2: Every constant used to produce the models of Table 1 . The first block specifies protective alignment of the original autoencoder; the second specifies tamper-resistance training starting from the aligned checkpoint. The simulated attacks used in tamper-resistance training are drawn from a pool of thirty-six configurations, one per epoch in the listed order, so a twenty-epoch run sees encoder-scope attacks first and decoder-tail attacks after.

Figure 4: The image the artist publishes, the trigger it carries, and the target that trigger activates, on two images. (a) the original artwork; (b) the protected version the artist publishes, carrying the trigger at $\epsilon=2/255$ ; (c) the difference between (a) and (b), amplified forty times so that it is visible at all; (d) the target the protective alignment is trained to produce from (b). Columns (a) and (b) are indistinguishable by design: over the 100 evaluation images the mean distortion between them is 45.1 dB PSNR, 0.990 SSIM and 0.010 LPIPS. The target in (d) is never published. It is what the deployed model produces in place of an edit, and each target renders the identifier of the image it is assigned to, which is why the two differ.

Objective descended	Approximation of the attack	$t{=}2$	$t{=}10$	$t{=}50$	Norm ratio
Equation 4	$K{=}3$ inner steps	0.249	0.165	0.049	0.020
	$K{=}5$ inner steps	0.252	0.168	0.051	0.044
	one-epoch refresh	0.188	0.195	0.102	0.590
Re-alignment	$K{=}3$ inner steps	0.246	0.162	0.047	0.023
	$K{=}5$ inner steps	0.254	0.169	0.050	0.051
	one-epoch refresh	0.218	0.258	0.163	0.748

Appendix

Table 3: How well each way of guessing the attacker’s position actually predicts it. Tamper-resistance training evaluates its objective at some approximation of where the attacker will be, and the two candidates are

K

differentiable inner steps from the current weights, or the offset reached by running the attack for one full epoch. Each row reports the cosine between that approximation’s parameter displacement and the displacement a real re-alignment attack has reached after

t

epochs, together with the ratio of their lengths at

t{=}10

. A cosine near zero means the approximation points somewhere else; a small norm ratio means it points the right way but has barely moved. The two blocks differ only in which objective the approximation descends; the ground truth is the same re-alignment attack in both.

	Protection (no attack)		Protection ( $t{=}20$ )		Fidelity
Model	LPIPS $\uparrow$	SSIM $\downarrow$	LPIPS $\uparrow$	SSIM $\downarrow$	LPIPS $\downarrow$
Original	0.225	0.750	–	–	–
Alignment	0.751	0.103	0.118	0.903	0.229
+ short-horizon	0.764	0.059	0.107	0.919	0.304
+ long-horizon	0.743	0.153	0.622	0.399	0.374

Appendix

Table 4: Additional protection and clean-fidelity metrics for the models in Table 1 . Protection is reported in LPIPS and SSIM between the edit of a protected image and the edit of its clean counterpart, so larger LPIPS and smaller SSIM both mean stronger protection. The fidelity column reports pre-attack LPIPS between an unprotected reconstruction and the original autoencoder’s, where smaller means less collateral damage to users who never asked for protection.

Configuration	Protection $\uparrow$ No attack	Protection $\uparrow$ $t{=}20$	Fidelity $\uparrow$ (dB)	Removal $\uparrow$ (epochs)
Full (long horizon)	0.743	0.622	25.7	$>$ 20
No epoch-long refresh (short horizon)	0.764	0.107	29.8	1
No anchors in $\mathcal{L}^{\psi}_{\mathrm{prot}}$	0.740	0.141	28.6	20
No target term in $\mathcal{L}^{\psi}_{\mathrm{prot}}$	0.731	0.114	25.9	10
No separation term $\mathcal{L}_{\mathrm{gap}}$	0.742	0.161	26.0	10
No clean fidelity at $\psi$ ( $\mathcal{L}^{\psi}_{\mathrm{fid}}$ )	0.665	0.462	22.3	$>$ 20

Appendix

Table 5: What each group of the tamper-resistance objective contributes. All models are trained for twenty epochs and evaluated under the same attack; ablations remove the epoch-long refresh or a loss component from Equation 2 . Protection is measured by LPIPS between the edit of a protected image and the edit of its clean counterpart, reported before the attack and after twenty epochs of it. Fidelity is measured by the PSNR of an unprotected reconstruction against the original autoencoder’s, measured before the attack. Removal denotes the first attack epoch at which protection LPIPS falls to or below the unaligned reference of

0.225

;

>20

indicates that the threshold was not reached within the first 20 attack epochs. The last two ablations illustrate that a delayed removal epoch can coincide with substantial loss of clean fidelity.

Figure 5: Every model we trained, placed in the plane of the two costs a provider pays. The horizontal axis is the protection surviving twenty epochs of attack, and the vertical axis is clean fidelity against the original autoencoder, so a point up and to the right is better on both. Filled points are the four that nothing else beats on both axes at once. Inside the shaded band the protection has fallen to the level of a model that was never aligned, and one ablation lies below the axis at 9.2 dB.

Checkpoint	Protection $\uparrow$ No attack	Protection $\uparrow$ $t{=}1$	Protection $\uparrow$ $t{=}20$	Fidelity $\uparrow$ (dB)	Removal $\uparrow$ (epochs)
Alignment	0.751	0.121	0.118	30.9	1
epoch 5	0.724	0.565	0.129	23.9	10
epoch 10	0.742	0.672	0.571	18.4	$>$ 20
epoch 15	0.732	0.457	0.468	24.2	$>$ 20
epoch 20	0.743	0.714	0.622	25.7	$>$ 20

Appendix

Table 6: The trade-off along one run of tamper-resistance training, sampled every five epochs. The first row is the aligned checkpoint the run starts from, so reading down the table shows what each further five epochs of training buys and costs. Protection LPIPS, clean-fidelity PSNR, and removal epochs follow the definitions in Table 5 . Protection is reported before attack and after one and twenty attack epochs, and clean fidelity is measured before attack.

Figure 6: The re-alignment attack on three images. Each pair is one protective alignment: the upper row of the pair received no tamper-resistance training (w/o TR) and the lower row did (w/ TR). Columns are attack epochs $t\in\{0,1,3,5,10,20\}$ under one seed, and the last column is an edit of an unprotected image under the same model. Without the training the protection is gone after a single epoch in all three cases, and the edit becomes indistinguishable from the clean-image edit beside it. With it the edit is still far from that clean-image edit at $t=20$ , although the pattern is no longer crisp. The last column is what a user who never asked for protection gets from the same model, while our clean-fidelity metric quantifies changes in reconstructions rather than edits.

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,169 papers

Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch
Mar 13, 2026Qichen Zhao, Shengfang Zhai, Xinjian Bai +4Adversarial Purification Adversarial Robustness
Peking University · National University of Singapore · The University of Western Australia
Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemes
Feb 25, 2026Xavier Pleimling, Sifat Muhammad Abdullah, Gunjan Balde +4Adversarial Attacks Adversarial Robustness
University of Texas at San Antonio San Antonio, TX, USA
Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks
May 26, 2026Kevin Kuo, Virginia Smith, Chhavi YadavLLM Fine-Tuning Adversarial Attacks on LLMs
Carnegie Mellon University · Simons Institute, UC Berkeley

Mar 13, 2026J/K move · Enter open · S save

Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch

Qichen Zhao, Shengfang Zhai, Xinjian Bai, Qingni Shen+3

Peking University · National University of Singapore · The University of Western Australia

Adversarial Purification Adversarial Robustness Diffusion Models Diffusion-Based Image Editing

Open paper AskPDF arXiv

Diffusion models enable high-fidelity image editing but can also be misused for unauthorized style imitation and harmful content generation. To mitigate these risks, proactive image protection methods embed small, often imperceptible adversarial perturbations into images before sharing to disrupt downstream editing or fine-tuning. However, in realistic post-release scenarios, content owners cannot control downstream processing pipelines, and protections optimized for a surrogate model may fail when attackers use mismatched diffusion pipelines. Existing purification methods can weaken protections but often sacrifice image quality and rarely examine architectural mismatch. We introduce a unified post-release purification framework to evaluate protection survivability under model mismatch. We propose two practical purifiers: VAE-Trans, which corrects protected images via latent-space projection, and EditorClean, which performs instruction-guided reconstruction with a Diffusion Transformer to exploit architectural heterogeneity. Both operate without access to protected images or defense internals. Across 2,100 editing tasks and six representative protection methods, EditorClean consistently restores editability. Compared to protected inputs, it improves PSNR by 3-6 dB and reduces FID by 50-70 percent on downstream edits, while outperforming prior purification baselines by about 2 dB PSNR and 30 percent lower FID. Our results reveal a purify-once, edit-freely failure mode: once purification succeeds, the protective signal is largely removed, enabling unrestricted editing. This highlights the need to evaluate protections under model mismatch and design defenses robust to heterogeneous attackers.

Abstract

Diffusion Models

Explore similar work

Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch

Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemes

Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks