WaterVIB: Learning Minimal Sufficient Watermark Representations via Variational Information Bottleneck

Haoyuan He; Yu Zheng; Jie Zhou; Jiwen Lu

cs.LGFeb 25, 2026

WaterVIB: Learning Minimal Sufficient Watermark Representations via Variational Information Bottleneck

Authors: Haoyuan He, Yu Zheng, Jie Zhou, Jiwen Lu

Organizations: Department of Automation, Tsinghua University.

Abstract

Robust watermarking is critical for intellectual property protection, whereas existing methods face a severe vulnerability against regeneration-based AIGC attacks. We identify that existing methods fail because they entangle the watermark with high-frequency cover texture, which is susceptible to being rewritten during generative purification. To address this, we propose WaterVIB, a theoretically grounded framework that reformulates the encoder as an information sieve via the Variational Information Bottleneck. Instead of overfitting to fragile cover details, our approach forces the model to learn a Minimal Sufficient Statistic of the message. This effectively filters out redundant cover nuances prone to generative shifts, retaining only the essential signal invariant to regeneration. We theoretically prove that optimizing this bottleneck is a necessary condition for robustness against distribution-shifting attacks. Extensive experiments demonstrate that WaterVIB significantly outperforms state-of-the-art methods, achieving superior zero-shot resilience against unknown diffusion-based editing.

Figures & tables

Figure 1: Vulnerability of standard watermarking versus the robustness of our VIB method. The residual visualizations (Top) reveal a strong correlation between the watermark signal and AIGC purification, highlighting the fragility of texture-entangled methods. We visualize the Bit Accuracy (higher is better) across six AIGC editing benchmarks. To clearly demonstrate the relative improvements on tasks with varying difficulty levels, each axis is independently normalized to its effective range

(a) Spectral Energy Distribution Alignment
Signal Component	High Freq.	Mid Freq.	Low Freq.	Dominant
Watermark Signal ( $\delta$ )	30.20%	45.45%	24.35%	Mid-High
AIGC Distortion ( $s_{atk}$ )	23.36%	20.49%	56.15%	Mid-High*
Natural Images ( $x$ )	1.16%	11.92%	86.91%	Low

(b) Structural Dependence (PCC with Cover Image)

Table 1: Evidence of Signal-Spatial Alignment. Top: Spectral energy distribution shows the AIGC distortion (

s_{atk}

) targets the same high-frequency bands as the watermark (

\delta

). Bottom: Pearson Correlation (PCC) shows the attack is structurally dependent on the cover image, unlike random noise.

Vector Component	Projection Value	Direction	Effect on Decoding
Gradient Direction ( $\nabla_{x}\mathcal{L}$ )	N/A	Target	N/A
Watermark Signal ( $\delta$ )	$-0.063$	Anti-Gradient	Minimizes Error
AIGC Distortion ( $s_{atk}$ )	$\mathbf{+0.027}$	Gradient	Maximizes Error

Table 2: Gradient Counter-Optimization Analysis. We project the signals onto the loss gradient direction

\nabla_{x}\mathcal{L}

. A negative projection implies loss minimization, while a positive projection implies loss maximization.

Figure 2: The WaterVIB Architecture. We propose a Stochastic Information Sieve mechanism (Part 2) to defend against generative purification (Part 1). By injecting noise via a learnable bottleneck layer, WaterVIB penalizes the retention of cover-specific details ( $I(Z;X)$ ) via the Information Bottleneck principle. This explicitly disentangles the watermark signal from the cover texture, yielding a stochastic representation $\mathbf{U}$ that is invariant to the semantic projections performed by diffusion models.

(I) Local Editing (BER ‰)
Attack Method	EditGuard	+VIB (Ours)	Red.
SD-Inpainting	0.35	0.03	91%
ControlNet-Inp.	0.13	0.14	-7%
SDXL-Refiner	0.30	0.03	90%
RePaint	0.26	0.08	69%
(II) Global Purification (BER %)

Table 3: Zero-shot Robustness against Generative Editing. Evaluation under (I) Local Editing and (II) Global Purification. The reported metric is Bit Error Rate (BER). Red. denotes the relative reduction in error rate.

Method	Payload	PSNR $\uparrow$	Clean ‰ $\downarrow$	Noise ‰ $\downarrow$
DWT-DCT	100 bits	38.1	0.56	10.84
TrustMark	100 bits	42.3	1.00	6.60
WM-A	30 bits	38.6	0.63	140.20
EditGuard	100 bits	40.4	0.05	3.21
+VIB (Ours)	100 bits	40.3	0.03	0.08

Table 4: Comparison with SOTA on Standard Distortions. Evaluation on EditGuard(+VIB) architecture. We report PSNR (dB) for imperceptibility and Bit Error Rate (BER %) for robustness.

Distortion Type	Baseline	+VIB(Ours)	Red.
Gaussian Noise‰	0.09	0.01	89%
JPEG ‰	9.61	0.40	96%
Poisson Noise‰	0.01	0.02	-
Cropout%	29.61	32.82	-
Dropout%	9.11	4.50	51%
Resize (Scaling)%	81.75	0.01	99.99%

Table 5: Detailed Robustness Analysis (EditGuard). Comparison of Bit Error Rate (BER %). Red. denotes the relative reduction in error rate.

(I) Visual Quality & Clean Accuracy
Metric	Baseline	+VIB(Ours)	Red.
PSNR (dB) $\uparrow$	36.5	37.0	-
SSIM $\uparrow$	0.97	0.97	-
Clean BER (%)	9.92	9.54	4%
(II) Standard Distortions
Crop (0.5)	10.37	9.76	6%

Table 6: Universality on Lightweight Models (HiDDeN). Evaluation on HiDDeN (30 bits). We report PSNR (dB) and Bit Error Rate (BER %). Red. denotes the relative reduction in error rate. Note the significant improvements across structural (Resize/Cropout) and adversarial attacks.

Figure 3: Feature Space Visualization (t-SNE). We visualize the latent embeddings of 10 random messages (different colors), each embedded into 20 cover images. In the Baseline (EditGuard), attacked samples (triangles) undergo significant feature drift, collapsing toward a shared manifold region regardless of their original message identity, which leads to high bit error rates. (b) With our WaterVIB, the drift paths (lines) are significantly reduced, and attacked samples remain anchored within the high-density clusters of their respective clean counterparts.

Method	EditGuard	+VIB (Ours)	Red.
Ratio $\rho$ $\downarrow$	0.4285	0.1167	73%

Table 7: Gradient Interference Analysis. Comparison of the Gradient Interference Ratio (

\rho

). Lower is better.

Figure 4: Generalization Gap Analysis. We evaluate the training dynamics by plotting the ratio of validation loss to training loss (

\mathcal{L}_{\text{val}}/\mathcal{L}_{\text{train}}

) across epochs. A ratio significantly greater than

1

indicates overfitting. The plots correspond to the Encoder (top), Decoder (middle), and Total Loss (bottom).

Panel of Figure 5 — Figure 5: Impact of $\beta$ on Robustness. We plot the BER under AIGC purification as a function of $\beta$ . The Baseline coresponds to $\beta=0$ . The curve reveals a distinct “sweet spot” at $\beta=0.00015$ .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Distortion Type	Parameter	Range / Value
Crop	Ratio	$[0.4,0.55]$
Cropout	Ratio	$[0.25,0.35]$
Dropout	Keep Prob.	$[0.65,0.75]$
Resize (Scaling)	Scale Factor	$[0.4,0.6]$
JPEG Compression	Coefficients	$(25,9,9)$

Appendix

Table 8: Configuration of Differentiable Noise Layers used in HiDDeN-VIB Training.

Method	Blender Dataset				LLFF Dataset
Method	BER $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	BER $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$
NeRF-Sig	0.00	57.17	$5\times 10^{-5}$	0.9998	0.10	46.51	0.0016	0.9973
+ WaterVIB	0.00	58.99	$3\times 10^{-5}$	0.9999	0.10	49.61	0.0009	0.9984

Appendix

Table 9: Comparison of watermarking imperceptibility on NeRF-Signature. We measure the quality of watermarked renderings against clean NeRF renderings. Metrics include Bit Error Rate (BER, lower is better), PSNR (higher is better), LPIPS (lower is better), and SSIM (higher is better).

Method	Rotation		Scaling		Blurring		Brightness		JPEG		Cropping		Average
Method	Acc	PSNR	Acc	PSNR	Acc	PSNR	Acc	PSNR	Acc	PSNR	Acc	PSNR	Acc	PSNR
NeRF-Sig	0.999	52.36	1.000	54.94	1.000	55.55	1.000	55.20	0.999	51.76	1.000	56.23	0.999	54.32
+ WaterVIB	0.997	53.26	1.000	55.76	1.000	57.07	1.000	55.98	0.998	51.93	1.000	57.54	0.999	55.26

Appendix

Table 10: Robustness evaluation on the Blender dataset (32 bits). We report the Bit Accuracy (ACC,

\uparrow

) and the PSNR (

\uparrow

) of the watermarked views. WaterVIB maintains high robustness while consistently improving visual quality.

Method	Cosine Sim. ( $\cos\theta$ )	Effective Proj. ( $\eta\cdot\|\cos\theta\|$ )
Baseline (EditGuard)	$-0.0395$	0.1572
WaterVIB (Ours)	0.0077	0.0306

Appendix

Table 11: Orthogonality Analysis. We compare the cosine similarity and the effective projection of the attack noise onto the watermark direction. Effective Proj. serves as the decisive metric for signal survival.

Noise Type	Cosine Sim. ( $\mathbf{s}\cdot\mathbf{x}$ )	Pearson Corr. (PCC)
Random Gaussian	$\approx 0.000$	$\approx 0.000$
AIGC (SDXL 1.0)	-0.1918	-0.3399

Appendix

Table 12: Correlation Analysis: AIGC Distortion vs. Cover Image. Unlike random noise (Gaussian) which is statistically orthogonal to the image content, AIGC distortion exhibits a significant negative correlation, indicating an active subtraction of image features.

Figure 7: Qualitative Comparison of Watermark Residuals. We visualize the absolute difference $|x_{wm}-x|$ (amplified for visibility) between the watermarked and original images. The Baseline (HiDDeN) residuals are heavily entangled with high-frequency image textures. In contrast, WaterVIB produces a more dispersed noise pattern, indicating that the Information Bottleneck constraint successfully decouples the watermark signal from fragile cover details.

Model	FP (Clean)	FN (Watermarked)	FN (WM + Noised)	KL-Div $\uparrow$
HiDDeN (Baseline)	0.00%	0.00%	38.50%	3.33
HiDDeN-VIB (Ours)	0.00%	0.00%	1.98%	3.84

Appendix

Table 13: Re-Embedding Defense Performance. Comparison of detection robustness. The VIB constraint significantly improves detection under noise, preventing unauthorized re-embedding.

Explore similar work

SHIFT: Stochastic Hidden-Trajectory Deflection for Removing Diffusion-based Watermark
Mar 31, 2026Rui Bao, Zheng Gao, Xiaoyu Li +3Watermark Removal Diffusion Model Watermarking
University of New South Wales · Griffith University
Exploring Weaknesses of Generative Image Watermarks against Latent Frequency Masking
Oct 1, 2026Kirill Aistov, Khaled Abud, Irina Serzhenko +8Diffusion Model Watermarking Watermark Removal
MSU AI Institute, Moscow, Russia · Trusted AI Research Center RAS, Moscow, Russia
DRIFT: Removing Diffusion Watermarks by Deflecting the Generative Trajectory
Sep 8, 2026Rui Bao, Zheng Gao, Xiaoyu Li +3Diffusion Model Watermarking Watermark Removal
University of New South Wales · Griffith University

Mar 31, 2026J/K move · Enter open · S save

SHIFT: Stochastic Hidden-Trajectory Deflection for Removing Diffusion-based Watermark

Rui Bao, Zheng Gao, Xiaoyu Li, Xiaoyan Feng+2

University of New South Wales · Griffith University

Watermark Removal Diffusion Model Watermarking Diffusion Models

Open paper AskPDF arXiv

Diffusion-based watermarking methods embed verifiable marks by manipulating the initial noise or the reverse diffusion trajectory. However, these methods share a critical assumption: verification can succeed only if the diffusion trajectory can be faithfully reconstructed. This reliance on trajectory recovery constitutes a fundamental and exploitable vulnerability. We propose

\underline{\mathbf{S}}

tochastic

\underline{\mathbf{Hi}}

dden-Trajectory De

\underline{\mathbf{f}}

lec

\underline{\mathbf{t}}

ion (

\mathbf{SHIFT}

), a training-free attack that exploits this common weakness across diverse watermarking paradigms. SHIFT leverages stochastic diffusion resampling to deflect the generative trajectory in latent space, making the reconstructed image statistically decoupled from the original watermark-embedded trajectory while preserving strong visual quality and semantic consistency. Extensive experiments on nine representative watermarking methods spanning noise-space, frequency-domain, and optimization-based paradigms show that SHIFT achieves 95%--100% attack success rates with nearly no loss in semantic quality, without requiring any watermark-specific knowledge or model retraining.

Abstract

Explore similar work

SHIFT: Stochastic Hidden-Trajectory Deflection for Removing Diffusion-based Watermark

Exploring Weaknesses of Generative Image Watermarks against Latent Frequency Masking

DRIFT: Removing Diffusion Watermarks by Deflecting the Generative Trajectory