SHIFT: Stochastic Hidden-Trajectory Deflection for Removing Diffusion-based Watermark

Rui Bao; Zheng Gao; Xiaoyu Li; Xiaoyan Feng; Yang Song; Jiaojiao Jiang

cs.CVMar 31, 2026

SHIFT: Stochastic Hidden-Trajectory Deflection for Removing Diffusion-based Watermark

Authors: Rui Bao, Zheng Gao, Xiaoyu Li, Xiaoyan Feng, Yang Song, Jiaojiao Jiang

Organizations: University of New South Wales · Griffith University

Abstract

Diffusion-based watermarking methods embed verifiable marks by manipulating the initial noise or the reverse diffusion trajectory. However, these methods share a critical assumption: verification can succeed only if the diffusion trajectory can be faithfully reconstructed. This reliance on trajectory recovery constitutes a fundamental and exploitable vulnerability. We propose $\underline{\mathbf{S}}$ tochastic $\underline{\mathbf{Hi}}$ dden-Trajectory De $\underline{\mathbf{f}}$ lec $\underline{\mathbf{t}}$ ion ( $\mathbf{SHIFT}$ ), a training-free attack that exploits this common weakness across diverse watermarking paradigms. SHIFT leverages stochastic diffusion resampling to deflect the generative trajectory in latent space, making the reconstructed image statistically decoupled from the original watermark-embedded trajectory while preserving strong visual quality and semantic consistency. Extensive experiments on nine representative watermarking methods spanning noise-space, frequency-domain, and optimization-based paradigms show that SHIFT achieves 95%--100% attack success rates with nearly no loss in semantic quality, without requiring any watermark-specific knowledge or model retraining.

Figures & tables

Figure 1: The overall framework of SHIFT

Method	Clean	Black-box [ 29 ]	Remove [ 18 ]	SHIFT
Highly vulnerable ( $\lambda\leq 0.15$ )
SEAL [ 2 ]	0	100	100	98
SFW [ 20 ]	0	100	100	100
WIND [ 1 ]	0	100	100	98
PRC [ 10 ]	0	100	100	97
Moderately robust ( $\lambda=0.30$ )

Table 1: Overall attack results. All values are ASR (%).

Figure 2: Comparison of

L_{1}

and

L_{2}

noise distances across nine watermarking methods.

Panel of Figure 3 — Figure 3: Qualitative comparison with different attack.

Method Group	CLIP score $\uparrow$	FID $\downarrow$
Black-box attack	31.877	106.831
Removing attack	31.356	96.431
Our attack	32.194	73.471

Table 2: Image quality comparison across attack methods.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Qualitative comparison of visual fidelity before and after attack across nine watermarking methods. In each subfigure, the left column shows the watermarked images and the right column shows the attacked images at attack strength $\lambda=0.50$ .

Panel of Figure 5 — Figure 5: Visual comparison of watermark robustness under different attack strengths. PRC, ROBIN, and Tree-Ring represent weakly, moderately, and highly robust watermarking methods, respectively. As $\lambda$ increases, the attacked outputs reveal clear differences in the fidelity–evasion trade-off across robustness levels.

Figure 6: Mean $L_{1}$ and $L_{2}$ noise distances as functions of attack strength $\lambda$ across nine watermarking methods. Both metrics increase monotonically with $\lambda$ , confirming that SHIFT progressively decouples the attacked image from the original watermark trajectory. The curves converge to a method-agnostic regime near the random Gaussian baseline at large $\lambda$ , consistent with the theoretical bound in Theorem A.7 .

Explore similar work

Sep 8, 2026

DRIFT: Removing Diffusion Watermarks by Deflecting the Generative Trajectory

Diffusion watermarking embeds verifiable signals into the generative process and commonly verifies them by recovering trajectory-dependent evidence, making the marks robust to conventional pixel-space distortions. Existing removal attacks either regenerate along deterministic trajectories, which often preserve the watermark-bearing latent structure, or optimize every image separately. We identify the reliance on a recoverable generative trajectory as a common attack surface among the schemes we study. Based on this observation, we propose DRIFT, a black-box attack that combines partial forward diffusion with stochastic reverse resampling. Forward re-noising limits source information available to a fixed-depth recovery pipeline, while stochastic reversal supplies alternative noise-driven paths whose removal benefit we isolate through matched sampler comparisons. Adaptive DRIFT searches a selected ladder for each image's first verifier-rejected rung and refines fidelity while retaining only updates rejected by the same verifier. At fixed depth, we derive information-theoretic and Wasserstein source-dependence bounds; under realized-ladder monotonicity, the first rejected rung is least distorted among rejected rungs on that ladder, and verifier-gated refinement preserves rejection. Across nine watermarks spanning three paradigms, DRIFT achieves 98-100% attack success and the best image quality among the compared attacks, without secret keys, verifier internals, or per-image gradient optimization.

Rui Bao, Zheng Gao, Xiaoyu Li +3

University of New South Wales · Griffith University

Diffusion Model Watermarking Watermark Removal

Ask PDF

Oct 9, 2026

WaveMark: Robust Wavelet-Domain Image Watermarking for Diffusion Models

Image watermarking enables provenance tracing for photorealistic images generated by diffusion models. However, state-of-the-art (SOTA) removal attacks can suppress embedded signals with little perceptual degradation. This vulnerability limits the effectiveness of watermarks for intellectual property protection and misuse attribution. We propose WaveMark, a novel wavelet-domain watermarking method that embeds messages in a diffusion model's initial noise. To support robust recovery from perturbed carriers, WaveMark distributes keyed replicas of message bits across spatial positions, wavelet orientations, and scales. Each carrier groups coefficients from four latent channels and encodes a randomized bit through magnitude projection, providing an explicit decoding margin. We evaluate WaveMark with SD2.1 on SDP and MS-COCO Captions under 14 attacks. WaveMark achieves 100.0% bit accuracy (BA) without attacks, while maintaining image quality comparable to the evaluated baselines. Under SOTA MarkNull and MarkNull-A, it retains over 88% BA, outperforming all baselines.

Jie Cao, Qi Li, Chaklam Cheong +4

Department of Electrical and Computer Engineering, Queen’s University, Kingston, Canada · School of Computing, Queen’s University, Kingston, Canada · School of Computer Science, Nanjing University, Jiangsu, China

Diffusion Model Watermarking Robust Watermarking

Ask PDF

Apr 24, 2026

Breaking Watermarks in the Frequency Domain: A Modulated Diffusion Attack Framework

Digital image watermarking has advanced rapidly for copyright protection of generative AI, yet the comparatively limited progress in watermark attack techniques has broken the attack-defense balance and hindered further advances in the field. In this paper, we propose FMDiffWA, a frequency-domain modulated diffusion framework for watermark attacks. Specifically, we introduce a frequency-domain watermark modulation (FWM) module and incorporate it into the sampling stages both the forward and reverse diffusion processes. This mechanism enables selective modulation of watermark-related frequency components, thereby allowing FMDiffWA to effectively neutralize the invisible watermark signals while preserving the perceptual quality of the attacked watermarked images. To achieve a better trade-off between attack efficacy and visual fidelity, we reformulate the training strategy of conventional diffusion models by augmenting the canonical noise estimation objective with an auxiliary refinement constraint. Comprehensive experiments demonstrate that FMDiffWA achieves superior visual fidelity compared to existing watermark attacks, while exhibiting strong generalization across diverse watermarking schemes.

Chunpeng Wang, Binyan Qu, Xiaoyu Wang +4

Qilu University of Technology (Shandong Academy of Sciences) Jinan, Shandong Province, China · Dalian Maritime University Dalian, Liaoning Province, China · Nanjing University of Science and Technology Nanjing, Jiangsu Province, China +1

Watermark Removal Diffusion Models

Ask PDF