cs.CROct 12, 2026
SaveBeyond User-Side Defenses: Image Protection in Open-Weight Diffusion Models
Organizations: Department of Electrical and Computer Engineering, Inha University
Abstract
Model providers can support image protection, but open-weight deployment introduces practical constraints. We propose four requirements: protection effectiveness, clean fidelity, compatibility, and tamper resistance. We incorporate long-horizon attack simulations to strengthen resistance to protection removal. Experiments with a single protection tool show improved resistance to re-alignment at a cost to clean fidelity, motivating further research toward practical model-side protection.
Figures & tables
Figure 1: Maintaining image protection after open-weight release. A re-alignment attack removes the protective behavior of an aligned model (top), whereas tamper-resistance training helps preserve protection under the same attack (bottom). Each row shows edits of the same protected image as the attack progresses, with the corresponding clean-image edit on the right. : attack epochs.
| Model | Protection | Fidelity (dB) | Removal (epochs) | |
|---|---|---|---|---|
| No Attack | Attacked | |||
| Original | 0.225 | N/A | N/A | N/A |
| Alignment | 0.751 | 0.118 | 30.9 | 1 |
| + short | 0.764 | 0.107 | 29.8 | 1 |
| + long | 0.743 | 0.622 | 25.7 | |
Table 1: Protection effectiveness, clean fidelity, and tamper resistance. Protection is measured by LPIPS before attack and after 20 epochs of encoder re-alignment. Fidelity is pre-attack reconstruction PSNR relative to the original autoencoder. Removal denotes the first attack epoch at which protection LPIPS falls to or below the unaligned reference of 0.225; means the threshold was not reached.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Quantity | Value |
|---|---|
| Protective Alignment | |
| , , , | 0.5, 0.5, 0.3, 2.0 |
| , , | 0.15, 3.0, 1.0 |
| optimizer, epochs | AdamW, lr , 80 |
| trainable | encoder, quant. convs, decoder tail |
| Tamper-Resistance Training | |
Table 2: Every constant used to produce the models of Table 1 . The first block specifies protective alignment of the original autoencoder; the second specifies tamper-resistance training starting from the aligned checkpoint. The simulated attacks used in tamper-resistance training are drawn from a pool of thirty-six configurations, one per epoch in the listed order, so a twenty-epoch run sees encoder-scope attacks first and decoder-tail attacks after.
Figure 4: The image the artist publishes, the trigger it carries, and the target that trigger activates, on two images. (a) the original artwork; (b) the protected version the artist publishes, carrying the trigger at ; (c) the difference between (a) and (b), amplified forty times so that it is visible at all; (d) the target the protective alignment is trained to produce from (b). Columns (a) and (b) are indistinguishable by design: over the 100 evaluation images the mean distortion between them is 45.1 dB PSNR, 0.990 SSIM and 0.010 LPIPS. The target in (d) is never published. It is what the deployed model produces in place of an edit, and each target renders the identifier of the image it is assigned to, which is why the two differ.
| Objective descended | Approximation of the attack | Norm ratio | |||
|---|---|---|---|---|---|
| Equation 4 | inner steps | 0.249 | 0.165 | 0.049 | 0.020 |
| inner steps | 0.252 | 0.168 | 0.051 | 0.044 | |
| one-epoch refresh | 0.188 | 0.195 | 0.102 | 0.590 | |
| Re-alignment | inner steps | 0.246 | 0.162 | 0.047 | 0.023 |
| inner steps | 0.254 | 0.169 | 0.050 | 0.051 | |
| one-epoch refresh | 0.218 | 0.258 | 0.163 | 0.748 |
Table 3: How well each way of guessing the attacker’s position actually predicts it. Tamper-resistance training evaluates its objective at some approximation of where the attacker will be, and the two candidates are differentiable inner steps from the current weights, or the offset reached by running the attack for one full epoch. Each row reports the cosine between that approximation’s parameter displacement and the displacement a real re-alignment attack has reached after epochs, together with the ratio of their lengths at . A cosine near zero means the approximation points somewhere else; a small norm ratio means it points the right way but has barely moved. The two blocks differ only in which objective the approximation descends; the ground truth is the same re-alignment attack in both.
| Protection (no attack) | Protection ( ) | Fidelity | |||
|---|---|---|---|---|---|
| Model | LPIPS | SSIM | LPIPS | SSIM | LPIPS |
| Original | 0.225 | 0.750 | – | – | – |
| Alignment | 0.751 | 0.103 | 0.118 | 0.903 | 0.229 |
| + short-horizon | 0.764 | 0.059 | 0.107 | 0.919 | 0.304 |
| + long-horizon | 0.743 | 0.153 | 0.622 | 0.399 | 0.374 |
Table 4: Additional protection and clean-fidelity metrics for the models in Table 1 . Protection is reported in LPIPS and SSIM between the edit of a protected image and the edit of its clean counterpart, so larger LPIPS and smaller SSIM both mean stronger protection. The fidelity column reports pre-attack LPIPS between an unprotected reconstruction and the original autoencoder’s, where smaller means less collateral damage to users who never asked for protection.
| Configuration | Protection No attack | Protection | Fidelity (dB) | Removal (epochs) |
|---|---|---|---|---|
| Full (long horizon) | 0.743 | 0.622 | 25.7 | 20 |
| No epoch-long refresh (short horizon) | 0.764 | 0.107 | 29.8 | 1 |
| No anchors in | 0.740 | 0.141 | 28.6 | 20 |
| No target term in | 0.731 | 0.114 | 25.9 | 10 |
| No separation term | 0.742 | 0.161 | 26.0 | 10 |
| No clean fidelity at ( ) | 0.665 | 0.462 | 22.3 | 20 |
Table 5: What each group of the tamper-resistance objective contributes. All models are trained for twenty epochs and evaluated under the same attack; ablations remove the epoch-long refresh or a loss component from Equation 2 . Protection is measured by LPIPS between the edit of a protected image and the edit of its clean counterpart, reported before the attack and after twenty epochs of it. Fidelity is measured by the PSNR of an unprotected reconstruction against the original autoencoder’s, measured before the attack. Removal denotes the first attack epoch at which protection LPIPS falls to or below the unaligned reference of ; indicates that the threshold was not reached within the first 20 attack epochs. The last two ablations illustrate that a delayed removal epoch can coincide with substantial loss of clean fidelity.
Figure 5: Every model we trained, placed in the plane of the two costs a provider pays. The horizontal axis is the protection surviving twenty epochs of attack, and the vertical axis is clean fidelity against the original autoencoder, so a point up and to the right is better on both. Filled points are the four that nothing else beats on both axes at once. Inside the shaded band the protection has fallen to the level of a model that was never aligned, and one ablation lies below the axis at 9.2 dB.
| Checkpoint | Protection No attack | Protection | Protection | Fidelity (dB) | Removal (epochs) |
|---|---|---|---|---|---|
| Alignment | 0.751 | 0.121 | 0.118 | 30.9 | 1 |
| epoch 5 | 0.724 | 0.565 | 0.129 | 23.9 | 10 |
| epoch 10 | 0.742 | 0.672 | 0.571 | 18.4 | 20 |
| epoch 15 | 0.732 | 0.457 | 0.468 | 24.2 | 20 |
| epoch 20 | 0.743 | 0.714 | 0.622 | 25.7 | 20 |
Table 6: The trade-off along one run of tamper-resistance training, sampled every five epochs. The first row is the aligned checkpoint the run starts from, so reading down the table shows what each further five epochs of training buys and costs. Protection LPIPS, clean-fidelity PSNR, and removal epochs follow the definitions in Table 5 . Protection is reported before attack and after one and twenty attack epochs, and clean fidelity is measured before attack.
Figure 6: The re-alignment attack on three images. Each pair is one protective alignment: the upper row of the pair received no tamper-resistance training (w/o TR) and the lower row did (w/ TR). Columns are attack epochs under one seed, and the last column is an edit of an unprotected image under the same model. Without the training the protection is gone after a single epoch in all three cases, and the edit becomes indistinguishable from the clean-image edit beside it. With it the edit is still far from that clean-image edit at , although the pattern is no longer crisp. The last column is what a user who never asked for protection gets from the same model, while our clean-fidelity metric quantifies changes in reconstructions rather than edits.
More in this topic
Diffusion Models
Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.
See all 1,169 papersDiffusion models enable high-fidelity image editing but can also be misused for unauthorized style imitation and harmful content generation. To mitigate these risks, proactive image protection methods embed small, often imperceptible adversarial perturbations into images before sharing to disrupt downstream editing or fine-tuning. However, in realistic post-release scenarios, content owners cannot control downstream processing pipelines, and protections optimized for a surrogate model may fail when attackers use mismatched diffusion pipelines. Existing purification methods can weaken protections but often sacrifice image quality and rarely examine architectural mismatch. We introduce a unified post-release purification framework to evaluate protection survivability under model mismatch. We propose two practical purifiers: VAE-Trans, which corrects protected images via latent-space projection, and EditorClean, which performs instruction-guided reconstruction with a Diffusion Transformer to exploit architectural heterogeneity. Both operate without access to protected images or defense internals. Across 2,100 editing tasks and six representative protection methods, EditorClean consistently restores editability. Compared to protected inputs, it improves PSNR by 3-6 dB and reduces FID by 50-70 percent on downstream edits, while outperforming prior purification baselines by about 2 dB PSNR and 30 percent lower FID. Our results reveal a purify-once, edit-freely failure mode: once purification succeeds, the protective signal is largely removed, enabling unrestricted editing. This highlights the need to evaluate protections under model mismatch and design defenses robust to heterogeneous attackers.
Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemes
Advances in Generative AI (GenAI) have led to the development of various protection strategies to prevent the unauthorized use of images. These methods rely on adding imperceptible protective perturbations to images to thwart misuse such as style mimicry or deepfake manipulations. Although previous attacks on these protections required specialized, purpose-built methods, we demonstrate that this is no longer necessary. We show that off-the-shelf image-to-image GenAI models can be repurposed as generic ``denoisers" using a simple text prompt, effectively removing a wide range of protective perturbations. Across 8 case studies spanning 6 diverse protection schemes, our general-purpose attack not only circumvents these defenses but also outperforms existing specialized attacks while preserving the image's utility for the adversary. Our findings reveal a critical and widespread vulnerability in the current landscape of image protection, indicating that many schemes provide a false sense of security. We stress the urgent need to develop robust defenses and establish that any future protection mechanism must be benchmarked against attacks from off-the-shelf GenAI models. Code is available in this repository: https://github.com/mlsecviswanath/img2imgdenoiser
Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks
Recent defenses for safeguarding open-weight large language models (LLMs) are intended to prevent adversarial usage. Underlying these defenses is an assumption that new harmful behavior is learned through fine-tuning rather than elicited by jailbreaking the model. Yet, pretrained LLMs already encode substantial harmful knowledge across many domains, which raises an important question: can an adversary jailbreak safeguarded models, to achieve harmful usage without fine-tuning at all? In this paper, we show that open-weight safeguards are susceptible to simpler strategies that, despite being well known, have not been systematically evaluated against these safeguards. Specifically, we evaluate two low-cost attacks--abliteration and prefilling--that do not rely on gradient-based optimization. Across three harmfulness evaluation benchmarks (BeaverTails, HarmBench, and AdvBench), these attacks increase attack success rates against safeguarded open-weight models from below 10% to a range of 16%-96%. To mitigate this vulnerability, we introduce abliteration-resistant tuning (ART), which incorporates an abliteration-based objective into training. ART can be layered onto existing defenses and reduces the success rates of abliteration, prefilling, and their combination by 10%-20%. These findings indicate that the attack surface for open-weight models is broader than previously characterized, and that evaluations of safeguarding defenses should incorporate a more diverse set of attack strategies beyond adversarial fine-tuning.