We explore the adaptability of neural image watermarking models; the extent to which one watermarking method can be transformed into another through lightweight adaptation. We introduce a framework that enables encoder-decoder watermarking architectures to become modular and upgradable, supporting multiple watermarking styles without re-training full models. Adaptation is performed by attaching a compact parameter-efficient adapter and fine-tuning under a multi-domain residual alignment loss (spatial, gradient, and frequency) while preserving bit accuracy via a secret-loss term. We evaluate this approach across a suite of recent watermarking models, including TrustMark, InvisMark, RoSteALS, HiDDeN, and WAM, demonstrating that adapted encoders closely reproduce target residuals and that adapted decoders recover messages with near-optimal accuracy under standard augmentations and distortions. We show that cross-model adaptation, with applications in interoperable and upgradable watermarking, can be achieved with as few as 1-2M trainable parameters per component (e.g., the 2M decoder adapter used in our multi-decoder inference pipeline).
Figures & tables
Figure 1: Method overview. Top (Training): a frozen base watermarking pair (Ebase,Dbase) is augmented with compact trainable CWLoRA adapters ( ∼ 1–2M params, shown in orange inside the conv blocks) to produce (Eadapted,Dadapted) that mimic a frozen target pair (Etarget,Dtarget) . Only the CWLoRA adapters are trained, via a residual alignment loss Lrec (matching R~ to the target residual R ) and a BCE secret loss Lsec (recovering m after the target noise module Atgt ). Bottom (Inference): a single frozen Dbase and a stack of ∼ 2M-parameter CWLoRA adapters replace N full decoders; a selector (classifier- or ECC-based) picks the right CWLoRA for a query image, reducing loaded decoder size from ∼ 1.2 GB to ∼ 0.14 GB ( ∼8× ) while preserving near-oracle accuracy.
Figure 2: Adapter comparison for decoder-only adaptation. Bit accuracy under noise vs. number of parameters for decoder adaptation with different types of adapters. The base decoder is from the TrustMarkQ method.
Base Decoder
InvisMark
RoSteALS
TrustMarkQ
TrustMarkP
WAM
Clean
Noised
Clean
Noised
Clean
Noised
Clean
Noised
Clean
Noised
Original ( Dtarget )
0.954
0.928
0.995
0.969
1.000
0.958
0.995
0.908
1.000
0.890
InvisMark
–
–
0.994
0.959
0.988
0.910
0.906
0.731
0.990
0.925
RoSteALS
0.937
0.881
–
–
0.995
0.897
0.981
0.842
0.974
0.917
TrustMarkQ
0.959
0.909
0.995
0.967
–
–
0.991
0.885
0.983
0.905
ResNet-50 ImageNet
0.500
0.500
0.994
0.965
0.995
0.931
0.501
0.501
0.990
0.925
Table 1: Decoder-only adaptation across base decoders. The “Original” row reports the baseline Acc[Etarget,Dtarget] (clean/noised). Subsequent rows show Acc[Etarget,Dadapted] after adapting with different base decoders Dbase using CWLoRA-16. The bottom row lists the PSNR of the original target encoder, which can be a proxy for difficulty of decoder adaptation.
Figure 3: Visualization of residual alignment for encoder adaptation. The first column shows the input references: the original image and the base residual from TrustMarkQ. The remaining columns show the corresponding target and adapted residuals for each method.
Base
Target
PSNRtgt
PSNRad
ACCtgt
ACCad
TrustMarkQ
TrustMarkP
48.33
43.1
0.91
0.92
RoSteALS
28.42
30.2
0.97
0.99
InvisMark
47.37
44.2
0.93
0.98
WAM
36.96
38.2
0.89
0.90
TrustMarkP
TrustMarkQ
39.69
38.85
0.95
0.96
RoSteALS
28.42
29.78
0.97
0.99
Table 2: Encoder–decoder adaptation across multiple base methods. PSNRtgt : imperceptibility of the target encoder ( PSNR[Etarget] , dB). PSNRad : imperceptibility of adapted encoder ( PSNR[Eadapted] , dB). ACCtgt : noised bit accuracy of the target pipeline ( Acc[Etarget,Dtarget] ). ACCad : noised bit accuracy of adapted pipeline ( Acc[Eadapted,Dadapted] ). All accuracies are measured after applying target’s noise module. Clean accuracies consistently match the target across all settings (see Appendix).
Target
Method
PSNR[Eadapted]
Acc[Eadapted,Dadapted]
Acc[Etarget,Dtarget]
InvisMark
TrustMarkQ-Small (Full)
35.17
0.97
0.93
TrustMarkQ (CWLoRA)
44.20
0.98
0.93
TrustMarkP
TrustMarkQ-Small (Full)
37.07
0.91
0.91
TrustMarkQ (CWLoRA)
43.10
0.92
0.91
Table 3: CWLoRA on a strong base vs. full fine-tuning of a compact model. Both approaches use a comparable number of trainable parameters, but CWLoRA on TrustMarkQ preserves the base’s perceptual quality while full fine-tuning of TrustMarkQ-Small suffers a large PSNR drop at similar accuracy.
Target
Geometric ↑
Valuemetric ↑
Regeneration ↑
Spoofing ↓
BBAdv ↑
InvisMark
0.57 / 0.69
0.65 / 0.65
0.50 / 0.50
0.72 / 0.81
0.95 / 0.91
RoSteALS
0.52 / 0.52
0.97 / 0.95
0.89 / 0.79
0.99 / 0.96
1.00 / 0.97
TrustMarkP
0.70 / 0.68
0.68 / 0.69
0.51 / 0.51
0.85 / 0.76
0.63 / 0.56
WAM
0.84 / 0.83
0.68 / 0.75
0.51 / 0.50
0.89 / 0.96
0.94 / 0.98
Table 4: OOD and adversarial robustness. Each cell reports Adapted (Ours) / Target (Original) accuracy after the listed perturbation. Arrows indicate the direction of higher robustness. Adapted models broadly inherit the target’s OOD invariances.
Target
PSNR[Eadapted]
Acc[Eadapted,Dadapted]
Acc[Etarget,Dtarget]
InvisMark
44.12
0.98
0.93
RoSteALS
30.26
0.98
0.97
TrustMarkP
42.83
0.92
0.91
WAM
38.15
0.91
0.89
Table 5: Cross-dataset evaluation on AI-generated images. Adapted models (TrustMarkQ base) evaluated on the Defactify Image Dataset, outside the MIRFLICKR training distribution. Metrics remain stable, confirming that adaptation does not overfit to the training dataset.
Method
Acc (%)
Size (GB)
Time (ms)
Clean
Noised
Original+Oracle
96.70
86.17
1.19
1.35
Adapter+Oracle
97.23
86.19
0.14
0.62
Original+Classifier
96.30
85.66
1.29
1.79
Adapter+Classifier
96.87
85.73
0.24
1.07
Original+ECC
96.13
85.68
1.19
4.75
Table 6: Comparison of approaches for multi-decoder inference. The Oracle rows are the upper bound for each decoder set. Acc is full-message accuracy.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Encoder-decoder adaptation performance. The base method ( Ebase,Dbase ) is TrustMarkQ. Top-left: PSNR between the input and output of Eadapted ( PSNR[Eadapted] , blue), and between outputs of Eadapted and Etarget ( PSNR[Eadapted,Etarget] , green); the red dashed line is PSNR[Etarget] (between input and output of Etarget ). Top-right: Acc[Eadapted,Dtarget] (clean/noised) with red dashed baseline Acc[Etarget,Dtarget] . Bottom-left: Acc[Etarget,Dadapted] (clean/noised). Bottom-right: Acc[Eadapted,Dadapted] (clean/noised). All “noised” metrics use the target method’s noise set for training and evaluation.
Figure 5: Performance of residual-matching-only encoder adaptation. Setting μ2=0 results in higher PSNR values compared to Figure 4 , while surprisingly, maintaining high decodability by Dtarget .
Loss Coefficients
TrustMarkP
WAM
PSNR[Eadapted]
PSNR[Eadapted,Etarget]
PSNR[Eadapted]
PSNR[Eadapted,Etarget]
α=1
46.0
47.7
38.1
36.0
α=1,β=0.5
46.5
52.5
41.6
36.8
α=1,γ=0.5
46.0
51.0
37.9
36.5
α=1,δ=0.5
45.1
53.2
38.3
37.8
α=1,β=0.5,γ=0.5,δ=0.5
45.9
54.1
38.8
38.3
Appendix
Table 7: The effect of residual-matching loss terms. Based on the results, using all four loss terms from Equation 3 achieves the best overall performance.
SSIM ( ↑ ) / LPIPS ( ↓ )
TrustMarkP
RoSteALS
InvisMark
HiDDeN
WAM
Target (Original)
1.00 / 0.00
0.84 / 0.04
1.00 / 0.00
0.81 / 0.22
0.96 / 0.03
Adapted (Ours)
0.99 / 0.00
0.89 / 0.11
0.99 / 0.01
0.77 / 0.22
0.97 / 0.02
Appendix
Table 8: Perceptual quality metrics (SSIM and LPIPS) for original target encoders and our adapted encoders. The base encoder used for adaptation is TrustMarkQ. LPIPS is calculated using an AlexNet backbone pre-trained on ImageNet.
Figure 6: Feature Similarity between base and target decoders. Left: Peak CKA Similarity. Right: Average Best-Match CKA. TrustMarkQ and InvisMark exhibit the lowest similarity scores for InvisMark (0.91 Peak, 0.24 Average), yet TrustMarkQ serves as the best base decoder for adapting to InvisMark (Table 1 ). This demonstrates that strong representational overlap is not the sole factor for effective watermark transformation.
Target Method
Adaptation Setup
Epochs
Total Time (Hours)
TrustMarkQ
Base Training (Original)
250
59.6
RoSteALS
Encoder-Decoder
150
110.5
WAM
Encoder-Decoder
150
71.9
TrustMarkP
Encoder-Decoder
150
44.5
InvisMark
Encoder-Decoder
150
40.9
HiDDeN
Encoder-Decoder
150
38.8
Appendix
Table 9: Training runtime analysis across different adaptation setups using TrustMarkQ as the base method. All adaptation variants are fine-tuned for 150 epochs on a single A100 (80GB) GPU, whereas the original TrustMarkQ base model is trained for 250 epochs.
Method
Params (M)
Type
TrustMark P/Q
Encoder
8.6
CNN
Decoder
23.7
CNN
HiDDeN
Encoder
0.2
CNN
Decoder
0.2
CNN
RoSteALS
Encoder
81.8
VQGAN+CNN
Decoder
23.7
CNN
Appendix
Table 10: Encoder and decoder parameter counts (in millions) and architecture type for each method. The CWLoRA-16 adapter parameters are reported with respect to TrustMarkQ as the base method, matching the configuration used in our encoder–decoder adaptation experiments (Section 4.3 ).
Method
Clean
Noised
Model Size (GB)
Runtime (ms)
FA (%)
FD (%)
Acc (%)
FA (%)
FD (%)
Acc (%)
Original+Oracle
0.00
3.30
96.70
0.00
13.83
86.17
1.19
1.35
Adapter+Oracle
0.00
2.77
97.23
0.00
13.81
86.19
0.14
0.62
Original+Classifier
0.48
3.22
96.30
2.39
11.95
85.66
1.29
1.79
Adapter+Classifier
0.48
2.65
96.87
2.28
11.99
85.73
0.24
1.07
Original+ECC
3.86
0.01
96.13
14.25
0.07
85.68
1.19
4.75
Appendix
Table 11: False-assignment and false-decoding rates for multi-decoder inference. Same runs as Table 6 . FA = false assignment (wrong decoder), FD = false decoding (right decoder, but ECC or data bits fail), Acc = 100 - FA - FD.
Figure 7: Per-watermark comparison of approaches for multi-decoder inference. Comparison of multi-decoder framework implemented with different settings on images watermarked by each watermarking method.
Figure 8: Additional residual alignment examples for encoder adaptation. Images are from validation set of MIRFLICKR.