JND-Guided Neural Watermarking with Spatial Transformer Decoding for Screen-Capture Robustness

Jiayi Qin; Jingwei Li; Chuan Wu

cs.CVMar 24, 2026

JND-Guided Neural Watermarking with Spatial Transformer Decoding for Screen-Capture Robustness

Authors: Jiayi Qin, Jingwei Li, Chuan Wu

Organizations: School of Optical-Electrical and Computer Engineering, Zhejiang Gongshang University

Abstract

Screen-shooting robust watermarking aims to imperceptibly embed extractable information into host images such that the watermark survives the complex distortion pipeline of screen display and camera recapture. However, achieving high extraction accuracy while maintaining satisfactory visual quality remains an open challenge, primarily because the screen-shooting channel introduces severe and entangled degradations including Moiré patterns, color-gamut shifts, perspective warping, and sensor noise. In this paper, we present an end-to-end deep learning framework that jointly optimizes watermark embedding and extraction for screen-shooting robustness. Our framework incorporates three key innovations: (i) a comprehensive noise simulation layer that faithfully models realistic screen-shooting distortions -- notably including a physically-motivated Moiré pattern generator -- enabling the network to learn robust representations against the full spectrum of capture-channel noise through adversarial training; (ii) a Just Noticeable Distortion (JND) perceptual loss function that adaptively modulates watermark embedding strength by supervising the perceptual discrepancy between the JND coefficient map and the watermark residual, thereby concentrating watermark energy in perceptually insensitive regions to maximize visual quality; and (iii) two complementary automatic localization modules -- a semantic-segmentation-based foreground extractor for captured image rectification and a symmetric noise template mechanism for anti-cropping region recovery -- that enable fully automated watermark decoding under realistic deployment conditions. Extensive experiments demonstrate that our method achieves an average PSNR of 30.94~dB and SSIM of 0.94 on watermarked images while embedding 127-bit payloads.

Figures & tables

Layer	$k$	$s$	Channels	Input
input	–	–	6	–
conv1	3	1	$6\to 32$	input
conv2	3	2	$32\to 32$	conv1
conv3	3	2	$32\to 64$	conv2
conv4	3	2	$64\to 128$	conv3
conv5	3	2	$128\to 256$	conv4

Table 1: Architecture of the encoder network.

k

: kernel size;

s

: stride.

Layer	$k$	$s$	Channels	Input
STN Sub-network
conv1	3	2	$3\to 32$	image
conv2	3	2	$32\to 64$	conv1
conv3	3	2	$64\to 128$	conv2
fc0	–	–	320 000	flatten(conv3)
fc1	–	–	$320\text{k}\to 128$	fc0

Table 2: Architecture of the decoder network.

k

: kernel size;

s

: stride. “–” indicates non-convolutional layers.

Category	Parameter	Value
Network	Training iterations	175,000
	Batch size	16
	Watermark length $L$	127 bits
	Learning rate	$10^{-4}$
Loss weights	$\lambda_{1}^{\text{loss}}$ (MSE)	2
	$\lambda_{2}^{\text{loss}}$ (LPIPS)	1.5

Table 3: Key training hyperparameters and their values.

Network	$\eta$	PSNR (dB)	SSIM	LPIPS	BER (%)
NET1	–	19.73	0.737	0.125	0.13
NET2	1	22.86	0.772	0.109	0.14
	0.5	23.59	0.806	0.098	0.36
	0.2	25.00	0.845	0.075	0.49
NET3	1	30.94	0.942	0.048	2.86

Table 4: Ablation results for the JND loss function. BER

\varepsilon

is measured at 40 cm frontal screen-shooting distance.

Angle	NET4 $\varepsilon_{1}$	NET5 $\varepsilon_{2}$	$\varepsilon_{1}-\varepsilon_{2}$
$-75^{\circ}$	7.46	8.27	$-0.81$
$-60^{\circ}$	1.85	1.97	$-0.12$
$-45^{\circ}$	2.55	2.73	$-0.18$
$-30^{\circ}$	2.39	3.18	$-0.80$
$-15^{\circ}$	2.53	3.68	$-1.15$
$0^{\circ}$	1.09	1.20	$-0.10$

Table 5: Ablation results for Moiré noise simulation. BER difference

\varepsilon_{1}-\varepsilon_{2}

shows improvement.

Angle	Method	BER (%)	$p_{r}$
$0^{\circ}$	No localization	46.72	–
	w/o segmentation	46.70	0.144
	w/ segmentation	7.39	0.963
$-15^{\circ}$	No localization	44.54	–
	w/o segmentation	44.54	0.112
	w/ segmentation	8.58	0.953

Table 6: Captured image region localization results.

Crop ratio	Dir.	w/ anti-crop	w/o anti-crop
$\gamma=0.05$	Top	2.72	4.87
	Left	2.46	5.90
	Bottom	1.76	3.91
	Right	2.47	4.93
$\gamma=0.40$	Top	4.96	53.10
	Left	6.24	55.96

Table 7: Anti-cropping localization results (BER in %).

Angle	Distance	Redmi K30	Sony A6100
$0^{\circ}$	40 cm	1.09	1.42
$0^{\circ}$	60 cm	1.57	1.81
$0^{\circ}$	100 cm	2.51	1.46
$0^{\circ}$	200 cm	11.65	3.33
$-45^{\circ}$	40 cm	2.55	1.44
$-15^{\circ}$	40 cm	2.53	1.65

Table 8: Screen-shooting robustness (BER %) of the proposed scheme under various conditions with HKCT279Q display.

Method	Cropping ratio				Distance (cm)				Angle range ( ∘ )
Method	0.05	0.25	0.45	0.55	30–39	40–49	50–59	60–69	$-40$ – $55$	$-30$ – $40$	$-15$ – $25$	$+15$ – $25$	$+30$ – $40$	$+40$ – $55$
Nakamura Nakamura et al. (2004)	–	–	–	–	14.75	17.81	18.44	19.50	19.63	16.31	15.44	13.88	22.27	16.11
Pramila Pramila et al. (2012)	–	–	–	–	28.38	28.62	28.94	30.19	19.92	16.41	19.91	20.70	20.31	22.34
DTW Fang et al. (2021)	0.31	0.31	0.31	22.34	4.81	2.73	12.11	11.72	16.02	5.47	3.91	2.73	2.73	3.52
SSRW Fang et al. (2019)	–	–	–	–	–	0.78	1.08	2.64	1.08	1.75	3.42	1.56	5.95	5.17
TERA Fang et al. (2022)	–	–	–	–	2.54	7.03	3.71	6.84	14.46	7.03	7.05	5.27	11.52	23.52

Table 9: Robustness comparison (BER %) across different methods. “–” indicates excessively high BER precluding error correction.

Method	Bits	PSNR	SSIM	LPIPS
Nakamura Nakamura et al. (2004)	32	38.97	0.99	–
Pramila Pramila et al. (2012)	32	40.26	0.99	–
DTW Fang et al. (2021)	64	39.73	0.99	–
SSRW Fang et al. (2019)	64	42.30	–	–
Ours	127	31.77	0.91	0.05

Table 10: Visual quality comparison of watermarked images across different methods.

Stage	Time (ms)	Throughput (FPS)
Watermark Embedding	21.55	46.40
Region Localization	29.03	34.44
Watermark Extraction	13.43	74.46
Total	64.01	15.62

Table 11: Per-image runtime analysis.

Robust Watermarking

Robust watermarking embeds or verifies ownership and provenance signals that survive editing, compression, model changes, and deliberate removal.

See all 207 papers

ResGuard: Enhancing Robustness Against Known Original Attacks in Deep Watermarking
Apr 4, 2026Hanyi Wang, Han Fang, Yupeng Qiu +2Robust Watermarking Watermark Removal
Shanghai Jiao Tong University China · University of Science and Technology of China China · National University of Singapore Singapore
SpreadMark: Robust Image Watermarking via Spread-Spectrum Embedding
Aug 4, 2026Wei Song, Yuxin Cao, Zhenchang Xing +4Dataset Watermarking Robust Watermarking
University of New South Wales, Australia · National University of Singapore, Singapore · CSIRO’s Data61, Australia
TIACam: Text-Anchored Invariant Feature Learning with Auto-Augmentation for Camera-Robust Zero-Watermarking
Feb 21, 2026Abdullah All Tanvir, Agnibh Dasgupta, Xin ZhongRobust Watermarking Invariant Representation Learning
Department of Computer Science, University of Nebraska Omaha

Apr 4, 2026J/K move · Enter open · S save

ResGuard: Enhancing Robustness Against Known Original Attacks in Deep Watermarking

Hanyi Wang, Han Fang, Yupeng Qiu, Shilin Wang+1

Shanghai Jiao Tong University China · University of Science and Technology of China China · National University of Singapore Singapore

Robust Watermarking Watermark Removal

Open paper AskPDF arXiv

Deep learning-based image watermarking commonly adopts an "Encoder-Noise Layer-Decoder" (END) architecture to improve robustness against random channel distortions, yet it often overlooks intentional manipulations introduced by adversaries with additional knowledge. In this paper, we revisit this paradigm and expose a critical yet underexplored vulnerability: the Known Original Attack (KOA), where an adversary has access to multiple original-watermarked image pairs, enabling various targeted suppression strategies. We show that even a simple residual-based removal approach, namely estimating an embedding residual from known pairs and subtracting it from unseen watermarked images, can almost completely remove the watermark while preserving visual quality. This vulnerability stems from the insufficient image dependency of residuals produced by END frameworks, which makes them transferable across images. To address this, we propose ResGuard, a plug-and-play module that enhances KOA robustness by enforcing image-dependent embedding. Its core lies in a residual specificity enhancement loss, which encourages residuals to be tightly coupled with their host images and thus improves image dependency. Furthermore, an auxiliary KOA noise layer injects residual-style perturbations during training, allowing the decoder to remain reliable under stronger embedding inconsistencies. Integrated into existing frameworks, ResGuard boosts KOA robustness, improving average watermark extraction accuracy from 59.87% to 99.81%.

Abstract

Robust Watermarking

Explore similar work

ResGuard: Enhancing Robustness Against Known Original Attacks in Deep Watermarking

SpreadMark: Robust Image Watermarking via Spread-Spectrum Embedding

TIACam: Text-Anchored Invariant Feature Learning with Auto-Augmentation for Camera-Robust Zero-Watermarking