cs.CVMar 19, 2026
SaveTransferable Multi-Bit Watermarking Across Frozen Diffusion Models via Latent Consistency Bridges
Organizations: School of Computer Science, University College Dublin, Ireland · School of Computer Science and Statistics, Trinity College Dublin, Ireland · Department of Knowledge Engineering, University of Science, VNU-HCMC, Vietnam
Abstract
As generative AI advances, global governance frameworks increasingly mandate verifiable content provenance. However, existing watermarking techniques face a critical policy-to-technology disconnect: sampling-based methods require computationally prohibitive inversion, while fine-tuning approaches are tethered to specific model checkpoints, hindering standardized, cross-model oversight. To bridge this gap, we introduce DiffMark, a plug-and-play multi-bit watermarking framework. DiffMark embeds a persistent, learned perturbation into every denoising step of a frozen diffusion model, accumulating a recoverable signal in the final latent space. To enable efficient training through the frozen network, we utilize Latent Consistency Models (LCMs) as a differentiable training bridge. DiffMark achieves 64-bit extraction in a single 16.4 ms forward pass, which is a speed-up over inversion baselines. By enabling per-image key flexibility and cross-architecture transferability without retraining, DiffMark provides the practical, scalable technical tooling necessary to operationalize user accountability and enforce emerging AI governance mandates.
Figures & tables
Figure 1: Overview of DiffMark.
| Detection Accuracy | Generation Consistency | Generation Quality | ||||||||
| Method | Type | Plug &Play | Multi-bit | Bit Acc (Clean) | TPR@1%FPR (Clean) | [email protected]%FPR (Clean) | PSNR | LPIPS | FID | CLIP-FID |
| StegaStamp | Post Generation | ✗ | 100 bits | 0.9994 | 1.0 | 1.0 | 11.34 | 0.7171 | 54.82 | 10.26 |
| Stable Signature | Fine-tuning | ✗ | 48 bits | 0.9950 | 0.9999 | 0.9900 | 16.23 | 0.5164 | 46.81 | 4.61 |
| AquaLoRA | ✗ | 48 bits | 0.9355 | 0.9970 | 0.9910 | 20.59 | 0.4791 | 32.32 | 1.98 | |
| Tree-Ring | Sampling | ✓ | 0 bit | — | 1.0 | 1.0 | 11.02 | 0.7441 | 47.09 | 4.65 |
| RingID | ✓ | 11 bits | — | 1.0 | 1.0 | 10.74 | 0.7481 | 47.18 | 4.77 | |
Table 1: Quantitative comparison on DiffusionDB. “Plug&Play” indicates whether the method operates on a frozen, unmodified diffusion model. Detection accuracy is measured on clean (unattacked) watermarked images. Generation consistency metrics compare watermarked vs. unwatermarked images from the same noise. Best results among plug-and-play methods are bolded ; overall best are underlined.
Figure 2: Bit Error Rate (BER) distributions for fixed-key and random-key generation across methods. Each violin shows the per-image BER over images; wider regions indicate higher density.
Figure 3: Cross-model transferability of DiffMark. Bit accuracy on four unseen SD-family models after training exclusively on SD 1.5, with no per-model fine-tuning.
Figure 4: Decode latency comparison (per image, L40S GPU). Latencies are averaged over 100 images; the x-axis is on a log scale.
Figure 5: Clean identification scaling. Top-1 identification accuracy as a function of database size (number of registered users) without any attack.
| Attack | Type | StegaStamp | Stable Signature | AquaLoRA | Tree-Ring | RingID | Shallow Diffuse | DiffMark (Ours) |
| Bright | Distortion Attacks | 1.00 | 1.00 | 0.96 | 0.74 | 1.00 | 1.00 | 1.00 |
| Compress | 1.00 | 1.00 | 0.99 | 0.74 | 1.00 | 1.00 | 1.00 | |
| Contrast | 1.00 | 1.00 | 0.97 | 0.74 | 0.98 | 1.00 | 1.00 | |
| Erase | 1.00 | 1.00 | 0.96 | 0.51 | 1.00 | 1.00 | 1.00 | |
| RCrop | 0.39 | 1.00 | 0.91 | 0.03 | 0.01 | 0.00 | 0.00 | |
| Rotation | 0.00 | 0.65 | 0.00 | 0.16 | 1.00 | 0.00 | 0.00 |
Table 2: Robustness evaluation across 13 attacks on DiffusionDB ([email protected]%FPR).
Figure 6: LCM Step Ablation
Figure 7: Bit Length Ablation.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Comparison of watermarking paradigms for DMs. (a) Sampling-based methods, (b) Fine-tuning-based methods modify model weights for fast multi-bit detection, and (c) DiffMark (Ours) injects a learned perturbation at every step of a frozen UNet, enabling fast single-pass multi-bit detection without weight modification.
| Module | Component | Parameters |
| Encoder | Bit embeddings ( ) | 8,192 |
| Spatial basis ( ) | 262,144 | |
| Refinement convolutions | 24K | |
| Variational heads ( , ) | 580 | |
| Total | 295,265 | |
| Decoder | Input conv + 5 downsample blocks | 700K |
Table 3: Parameter breakdown of the encoder and decoder.
| Parameter | Value |
| Training steps | 10,000 |
| Batch size | 16 |
| Optimizer | AdamW |
| Learning rate (encoder / decoder) | / |
| LR schedule | Warmup (500 steps) + linear decay to |
| Gradient clip (encoder / decoder) | 5.0 / 1.0 |
Table 4: Training hyperparameters.
| Loss | Symbol | Weight |
| LCM reconstruction | 1.0 | |
| DDIM supervision | 1.0 | |
| Magnitude constraint | 5.0 (MSE variant) | |
| KL regularization | (warmup, 1K steps) | |
| Orthogonality | 0.1 | |
| PRVL | 1.5 |
Table 5: Loss weights for main training.
| Attack | ID | Mechanism | Strength Range |
| Rotation | Rotation | Rotate by angle | |
| Resized Crop | RCrop | Crop + resize | scale |
| Random Erasing | Erase | Zero-fill random patch | area |
| Brightness | Bright | Enhance brightness | factor |
| Contrast | Contrast | Enhance contrast | factor |
| Gaussian Blur | Blur | Gaussian blur kernel | size |
Table 6: Summary of distortion attacks used for robustness evaluation.
| Attack | ID | Strength Range |
| Regeneration attacks | ||
| VAE | Regen-VAE | quality |
| Diffusion | Regen-Diff | steps |
| Rinsing (2 ) | Rinse-2xDiff | steps |
| Adversarial attacks | ||
| KL-VAE (grey-box) | Adv-KLVAE8 | : |
Table 7: Summary of regeneration and adversarial attacks used for robustness evaluation.
| Detection Accuracy | Generation Consistency | Generation Quality | |||||||
| Method | Type | Plug &Play | Multi-bit | Bit Acc (Clean) | [email protected]%FPR (Clean) | PSNR | LPIPS | FID | CLIP-FID |
| DALL-E3 | |||||||||
| StegaStamp | Post Generation | ✗ | 100 bits | 0.9988 | 1.0 | 9.48 | 0.7715 | 113.54 | 23.92 |
| Stable Signature | Fine-tuning | ✗ | 48 bits | 0.9915 | 1.0 | 13.51 | 0.5812 | 96.51 | 9.43 |
| AquaLoRA | ✗ | 48 bits | 0.9124 | 0.9440 | 18.28 | 0.3120 | 70.00 | 4.98 | |
| Tree-Ring | Sampling | ✓ | 0 bit | — | 1.0 | 9.74 | 0.7643 | 98.59 | 9.97 |
Table 8: Quantitative comparison on DALL-E3 and MS-COCO. Best results among plug-and-play methods are bolded ; overall best are underlined.
Figure 9: Identification accuracy under attacks. Top-1 accuracy vs. database size for representative distortion (solid) and regeneration (dashed) attacks. Most attacks preserve identification at users.
Figure 10: Per-image BER as a function of Hamming distance from the training key .
Figure 11: Cross-model transferability of DiffMark under attack. [email protected]%FPR is reported for five SD-family models across eight distortion attacks (left) and six regeneration attacks (right). DiffMark is trained exclusively on SD 1.5; no per-model fine-tuning is applied to the four unseen models.
Figure 12: Qualitative cross-model transferability of DiffMark. Each column shares the same prompt; each row corresponds to a different SD-family model. DiffMark is trained exclusively on SD 1.5 and applied to four unseen models (SD 2.1, DreamShaper 8, Realistic Vision 5.1, OpenJourney v4) without any per-model fine-tuning.
| Attack | Type | StegaStamp | Stable Signature | AquaLoRA | Tree-Ring | RingID | Shallow Diffuse | DiffMark (Ours) |
| DALL-E3 | ||||||||
| Bright | Distortion Attacks | 1.00 | 1.00 | 0.89 | 0.64 | 1.00 | 1.00 | 1.00 |
| Compress | 1.00 | 0.98 | 0.93 | 0.78 | 1.00 | 1.00 | 1.00 | |
| Contrast | 1.00 | 1.00 | 0.87 | 0.73 | 1.00 | 1.00 | 1.00 | |
| Erase | 1.00 | 1.00 | 0.88 | 0.53 | 1.00 | 1.00 | 1.00 | |
| RCrop | 0.37 | 1.00 | 0.82 | 0.03 | 0.00 | 0.00 | 0.00 | |
Table 9: Robustness comparison on DALL-E3 and MS-COCO under 13 attack types ([email protected]%FPR). Best results are bolded .
Figure 13: Qualitative examples of watermarked images generated by DiffMark with their corresponding input prompts. The embedded watermark is visually imperceptible.
Figure 14: Watermark signal analysis. Columns from left to right: watermarked image, clean image (same and prompt), difference map ( amplified), and delta heatmap (L2 norm across latent channels). The watermark perturbation is imperceptible in pixel space while maintaining spatially uniform energy in latent space.
Figure 15: Training progression of the watermark signal. Top row: heatmaps; bottom row: difference maps. Left to right: early training (pre-PRVL), mid training (post-PRVL), and late training (near convergence). The PRVL regularizer enforces spatial uniformity in , while pushes pixel-space differences toward high-frequency regions.
Figure 16: Per-channel decomposition. Left: overall L2 norm across channels. Right four panels: individual latent channels visualized with a diverging colormap. The watermark signal is distributed across all four channels with distinct spatial patterns.
Figure 17: Frequency domain analysis of the signal. (a) Average log power spectrum across all samples and channels, with DC at center. (b) Radial power profile showing suppressed low-frequency energy below the configured radius (red dashed line). (c) Single-example power spectrum for channel 0.
Figure 18: Difference map overlay on watermarked images. Top row: watermarked images. Bottom row: same images with the mean absolute difference rendered as a semi-transparent heatmap. Bright regions indicate stronger watermark perturbation, which concentrates at edges and textures.