We explore the adaptability of neural image watermarking models; the extent to which one watermarking method can be transformed into another through lightweight adaptation. We introduce a framework that enables encoder-decoder watermarking architectures to become modular and upgradable, supporting multiple watermarking styles without re-training full models. Adaptation is performed by attaching a compact parameter-efficient adapter and fine-tuning under a multi-domain residual alignment loss (spatial, gradient, and frequency) while preserving bit accuracy via a secret-loss term. We evaluate this approach across a suite of recent watermarking models, including TrustMark, InvisMark, RoSteALS, HiDDeN, and WAM, demonstrating that adapted encoders closely reproduce target residuals and that adapted decoders recover messages with near-optimal accuracy under standard augmentations and distortions. We show that cross-model adaptation, with applications in interoperable and upgradable watermarking, can be achieved with as few as 1-2M trainable parameters per component (e.g., the 2M decoder adapter used in our multi-decoder inference pipeline).
Figures & tables
Figure 1: Method overview. Top (Training): a frozen base watermarking pair (Ebase,Dbase) is augmented with compact trainable CWLoRA adapters ( ∼ 1–2M params, shown in orange inside the conv blocks) to produce (Eadapted,Dadapted) that mimic a frozen target pair (Etarget,Dtarget) . Only the CWLoRA adapters are trained, via a residual alignment loss Lrec (matching R~ to the target residual R ) and a BCE secret loss Lsec (recovering m after the target noise module Atgt ). Bottom (Inference): a single frozen Dbase and a stack of ∼ 2M-parameter CWLoRA adapters replace N full decoders; a selector (classifier- or ECC-based) picks the right CWLoRA for a query image, reducing loaded decoder size from ∼ 1.2 GB to ∼ 0.14 GB ( ∼8× ) while preserving near-oracle accuracy.
Figure 2: Adapter comparison for decoder-only adaptation. Bit accuracy under noise vs. number of parameters for decoder adaptation with different types of adapters. The base decoder is from the TrustMarkQ method.
Base Decoder
InvisMark
RoSteALS
TrustMarkQ
TrustMarkP
WAM
Clean
Noised
Clean
Noised
Clean
Noised
Clean
Noised
Clean
Noised
Original ( Dtarget )
0.954
0.928
0.995
0.969
1.000
0.958
0.995
0.908
1.000
0.890
InvisMark
–
–
0.994
0.959
0.988
0.910
0.906
0.731
0.990
0.925
RoSteALS
0.937
0.881
–
–
0.995
0.897
0.981
0.842
0.974
0.917
TrustMarkQ
0.959
0.909
0.995
0.967
–
–
0.991
0.885
0.983
0.905
ResNet-50 ImageNet
0.500
0.500
0.994
0.965
0.995
0.931
0.501
0.501
0.990
0.925
Table 1: Decoder-only adaptation across base decoders. The “Original” row reports the baseline Acc[Etarget,Dtarget] (clean/noised). Subsequent rows show Acc[Etarget,Dadapted] after adapting with different base decoders Dbase using CWLoRA-16. The bottom row lists the PSNR of the original target encoder, which can be a proxy for difficulty of decoder adaptation.
Figure 3: Visualization of residual alignment for encoder adaptation. The first column shows the input references: the original image and the base residual from TrustMarkQ. The remaining columns show the corresponding target and adapted residuals for each method.
Base
Target
PSNRtgt
PSNRad
ACCtgt
ACCad
TrustMarkQ
TrustMarkP
48.33
43.1
0.91
0.92
RoSteALS
28.42
30.2
0.97
0.99
InvisMark
47.37
44.2
0.93
0.98
WAM
36.96
38.2
0.89
0.90
TrustMarkP
TrustMarkQ
39.69
38.85
0.95
0.96
RoSteALS
28.42
29.78
0.97
0.99
Table 2: Encoder–decoder adaptation across multiple base methods. PSNRtgt : imperceptibility of the target encoder ( PSNR[Etarget] , dB). PSNRad : imperceptibility of adapted encoder ( PSNR[Eadapted] , dB). ACCtgt : noised bit accuracy of the target pipeline ( Acc[Etarget,Dtarget] ). ACCad : noised bit accuracy of adapted pipeline ( Acc[Eadapted,Dadapted] ). All accuracies are measured after applying target’s noise module. Clean accuracies consistently match the target across all settings (see Appendix).
Target
Method
PSNR[Eadapted]
Acc[Eadapted,Dadapted]
Acc[Etarget,Dtarget]
InvisMark
TrustMarkQ-Small (Full)
35.17
0.97
0.93
TrustMarkQ (CWLoRA)
44.20
0.98
0.93
TrustMarkP
TrustMarkQ-Small (Full)
37.07
0.91
0.91
TrustMarkQ (CWLoRA)
43.10
0.92
0.91
Table 3: CWLoRA on a strong base vs. full fine-tuning of a compact model. Both approaches use a comparable number of trainable parameters, but CWLoRA on TrustMarkQ preserves the base’s perceptual quality while full fine-tuning of TrustMarkQ-Small suffers a large PSNR drop at similar accuracy.
Target
Geometric ↑
Valuemetric ↑
Regeneration ↑
Spoofing ↓
BBAdv ↑
InvisMark
0.57 / 0.69
0.65 / 0.65
0.50 / 0.50
0.72 / 0.81
0.95 / 0.91
RoSteALS
0.52 / 0.52
0.97 / 0.95
0.89 / 0.79
0.99 / 0.96
1.00 / 0.97
TrustMarkP
0.70 / 0.68
0.68 / 0.69
0.51 / 0.51
0.85 / 0.76
0.63 / 0.56
WAM
0.84 / 0.83
0.68 / 0.75
0.51 / 0.50
0.89 / 0.96
0.94 / 0.98
Table 4: OOD and adversarial robustness. Each cell reports Adapted (Ours) / Target (Original) accuracy after the listed perturbation. Arrows indicate the direction of higher robustness. Adapted models broadly inherit the target’s OOD invariances.
Target
PSNR[Eadapted]
Acc[Eadapted,Dadapted]
Acc[Etarget,Dtarget]
InvisMark
44.12
0.98
0.93
RoSteALS
30.26
0.98
0.97
TrustMarkP
42.83
0.92
0.91
WAM
38.15
0.91
0.89
Table 5: Cross-dataset evaluation on AI-generated images. Adapted models (TrustMarkQ base) evaluated on the Defactify Image Dataset, outside the MIRFLICKR training distribution. Metrics remain stable, confirming that adaptation does not overfit to the training dataset.
Method
Acc (%)
Size (GB)
Time (ms)
Clean
Noised
Original+Oracle
96.70
86.17
1.19
1.35
Adapter+Oracle
97.23
86.19
0.14
0.62
Original+Classifier
96.30
85.66
1.29
1.79
Adapter+Classifier
96.87
85.73
0.24
1.07
Original+ECC
96.13
85.68
1.19
4.75
Table 6: Comparison of approaches for multi-decoder inference. The Oracle rows are the upper bound for each decoder set. Acc is full-message accuracy.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Encoder-decoder adaptation performance. The base method ( Ebase,Dbase ) is TrustMarkQ. Top-left: PSNR between the input and output of Eadapted ( PSNR[Eadapted] , blue), and between outputs of Eadapted and Etarget ( PSNR[Eadapted,Etarget] , green); the red dashed line is PSNR[Etarget] (between input and output of Etarget ). Top-right: Acc[Eadapted,Dtarget] (clean/noised) with red dashed baseline Acc[Etarget,Dtarget] . Bottom-left: Acc[Etarget,Dadapted] (clean/noised). Bottom-right: Acc[Eadapted,Dadapted] (clean/noised). All “noised” metrics use the target method’s noise set for training and evaluation.
Figure 5: Performance of residual-matching-only encoder adaptation. Setting μ2=0 results in higher PSNR values compared to Figure 4 , while surprisingly, maintaining high decodability by Dtarget .
Loss Coefficients
TrustMarkP
WAM
PSNR[Eadapted]
PSNR[Eadapted,Etarget]
PSNR[Eadapted]
PSNR[Eadapted,Etarget]
α=1
46.0
47.7
38.1
36.0
α=1,β=0.5
46.5
52.5
41.6
36.8
α=1,γ=0.5
46.0
51.0
37.9
36.5
α=1,δ=0.5
45.1
53.2
38.3
37.8
α=1,β=0.5,γ=0.5,δ=0.5
45.9
54.1
38.8
38.3
Appendix
Table 7: The effect of residual-matching loss terms. Based on the results, using all four loss terms from Equation 3 achieves the best overall performance.
SSIM ( ↑ ) / LPIPS ( ↓ )
TrustMarkP
RoSteALS
InvisMark
HiDDeN
WAM
Target (Original)
1.00 / 0.00
0.84 / 0.04
1.00 / 0.00
0.81 / 0.22
0.96 / 0.03
Adapted (Ours)
0.99 / 0.00
0.89 / 0.11
0.99 / 0.01
0.77 / 0.22
0.97 / 0.02
Appendix
Table 8: Perceptual quality metrics (SSIM and LPIPS) for original target encoders and our adapted encoders. The base encoder used for adaptation is TrustMarkQ. LPIPS is calculated using an AlexNet backbone pre-trained on ImageNet.
Figure 6: Feature Similarity between base and target decoders. Left: Peak CKA Similarity. Right: Average Best-Match CKA. TrustMarkQ and InvisMark exhibit the lowest similarity scores for InvisMark (0.91 Peak, 0.24 Average), yet TrustMarkQ serves as the best base decoder for adapting to InvisMark (Table 1 ). This demonstrates that strong representational overlap is not the sole factor for effective watermark transformation.
Target Method
Adaptation Setup
Epochs
Total Time (Hours)
TrustMarkQ
Base Training (Original)
250
59.6
RoSteALS
Encoder-Decoder
150
110.5
WAM
Encoder-Decoder
150
71.9
TrustMarkP
Encoder-Decoder
150
44.5
InvisMark
Encoder-Decoder
150
40.9
HiDDeN
Encoder-Decoder
150
38.8
Appendix
Table 9: Training runtime analysis across different adaptation setups using TrustMarkQ as the base method. All adaptation variants are fine-tuned for 150 epochs on a single A100 (80GB) GPU, whereas the original TrustMarkQ base model is trained for 250 epochs.
Method
Params (M)
Type
TrustMark P/Q
Encoder
8.6
CNN
Decoder
23.7
CNN
HiDDeN
Encoder
0.2
CNN
Decoder
0.2
CNN
RoSteALS
Encoder
81.8
VQGAN+CNN
Decoder
23.7
CNN
Appendix
Table 10: Encoder and decoder parameter counts (in millions) and architecture type for each method. The CWLoRA-16 adapter parameters are reported with respect to TrustMarkQ as the base method, matching the configuration used in our encoder–decoder adaptation experiments (Section 4.3 ).
Method
Clean
Noised
Model Size (GB)
Runtime (ms)
FA (%)
FD (%)
Acc (%)
FA (%)
FD (%)
Acc (%)
Original+Oracle
0.00
3.30
96.70
0.00
13.83
86.17
1.19
1.35
Adapter+Oracle
0.00
2.77
97.23
0.00
13.81
86.19
0.14
0.62
Original+Classifier
0.48
3.22
96.30
2.39
11.95
85.66
1.29
1.79
Adapter+Classifier
0.48
2.65
96.87
2.28
11.99
85.73
0.24
1.07
Original+ECC
3.86
0.01
96.13
14.25
0.07
85.68
1.19
4.75
Appendix
Table 11: False-assignment and false-decoding rates for multi-decoder inference. Same runs as Table 6 . FA = false assignment (wrong decoder), FD = false decoding (right decoder, but ECC or data bits fail), Acc = 100 - FA - FD.
Figure 7: Per-watermark comparison of approaches for multi-decoder inference. Comparison of multi-decoder framework implemented with different settings on images watermarked by each watermarking method.
Figure 8: Additional residual alignment examples for encoder adaptation. Images are from validation set of MIRFLICKR.
As generative models enable rapid creation of high-fidelity images, societal concerns about misinformation and authenticity have intensified. A promising remedy is multi-bit image watermarking, which embeds a multi-bit message into an image so that a verifier can later detect whether the image is generated by someone and further identify the source by decoding the embedded message. Existing approaches often fall short in capacity, resilience to common image distortions, and theoretical justification. To address these limitations, we propose ADD (Add, Dot, Decode), a multi-bit image watermarking method with two stages: learning a watermark to be linearly combined with the multi-bit message and added to the image, and decoding through inner products between the watermarked image and the learned watermark. On the standard MS-COCO benchmark, we demonstrate that for the challenging task of 48-bit watermarking, ADD achieves 100% decoding accuracy, with performance dropping by at most 2% under a wide range of image distortions, substantially smaller than the 14% average drop of state-of-the-art methods. In addition, ADD achieves substantial computational gains, with 2-fold faster embedding and 7.4-fold faster decoding than the fastest existing method. We further provide a theoretical analysis explaining why the learned watermark and the corresponding decoding rule are effective.
Language model watermarking schemes fall into two broad categories: inference-time methods, which modify the decoding process at generation time, and model-embedded methods, which encode a secret signal directly into the model weights. Inference-time watermarking methods incur additional inference overhead and are not applicable in the increasingly prevalent open-weight model settings. In contrast, model-embedded watermarks introduce no additional inference latency-generation uses the standard sampling pipeline-and are particularly well-suited to open-weight settings. However, existing model-embedded approaches such as GaussMark face a fundamental quality-detectability trade-off: achieving strong detection power typically requires weight perturbations that noticeably degrade generation quality. Our key insight is that regularizing toward a high-quality reference distribution decouples quality preservation from proximity to the base model, enabling a better quality-detectability trade-off. Building on this insight, we introduce MarkTune, a theoretically grounded on-policy fine-tuning framework that uses the GaussMark detection statistic as a reward while preserving generation quality through cross-entropy on high-quality human text or KL divergence to a stronger oracle model. Empirically, MarkTune substantially improves the quality-detectability frontier of GaussMark, approaching the detectability of strong inference-time schemes while preserving generation quality and downstream task performance. Marktune also remains robust under the evaluated text-editing and open-weight attacks and generalizes across datasets, retaining substantial detection power on corpora unseen during training. Code is available at https://github.com/zhyzmath/MarkTune-LLM-Watermarking.
Yizhou Zhao, Zhiwei Steven Wu, Adam Block
University of Pennsylvania · Carnegie Mellon University · Columbia University
Deep learning-based image watermarking, while robust against conventional distortions, remains vulnerable to advanced adversarial and regeneration attacks. Conventional countermeasures, which jointly optimize the encoder and decoder via a noise layer, face 2 inevitable challenges: (1) decrease of clean accuracy due to decoder adversarial training and (2) limited robustness due to simultaneous training of all three advanced attacks. To overcome these issues, we propose AdvMark, a novel two-stage fine-tuning framework that decouples the defense strategies. In stage 1, we address adversarial vulnerability via a tailored adversarial training paradigm that primarily fine-tunes the encoder while only conditionally updating the decoder. This approach learns to move the image into a non-attackable region, rather than modifying the decision boundary, thus preserving clean accuracy. In stage 2, we tackle distortion and regeneration attacks via direct image optimization. To preserve the adversarial robustness gained in stage 1, we formulate a principled, constrained image loss with theoretical guarantees, which balances the deviation from cover and previous encoded images. We also propose a quality-aware early-stop to further guarantee the lower bound of visual quality. Extensive experiments demonstrate AdvMark outperforms with the highest image quality and comprehensive robustness, i.e. up to 29%, 33% and 46% accuracy improvement for distortion, regeneration and adversarial attacks, respectively.
Jiahui Chen, Zehang Deng, Zeyu Zhang +3
Tsinghua University · Swinburne University of Technology · The Australian National University +2