We introduce OpenMarkX, an extensible platform for watermarking open-source model and agent outputs under standard inference. It takes models, compatible methods, private keys, and evaluation settings, and produces watermarked weights, detection results, and quality and robustness reports. Its 16 text-watermark configurations are complemented by agent, vision-language, image, and physics-field workflows. Our proposed method, OpenMark, addresses learnability and robustness through joint model--policy learning and perturbation-aware training. On several open-source LLMs, OpenMark achieves 0.92 TPR at 1% FPR while reducing perplexity by 20.5% on average relative to the strongest prior method, and remains more detectable after such modifications, e.g., retaining 0.57 TPR at 5% FPR after 0.5-ratio model merging, compared with at most 0.34 for prior methods.
Figures & tables
Figure 1: Overview of OpenMarkX . Configurable models and watermark methods feed embedding, generation, and evaluation. OpenMark supplies joint learning and perturbation-aware training.
Figure 2: Text watermarking for (a) closed-source and (b) open-source LLMs. Closed-source watermarking relies on watermark decoding, while open-source watermarking requires embedding the watermark into the model weights so that standard decoding still produces watermarked text.
Figure 3: (Left) learning-based and decoding-based KGW watermarks under varying watermark hyperparameters n and δ . (Right) existing learning-based watermarks’ AUC after merging with the unwatermarked LLM.
Figure 4: Overview of our proposed method. CAWP ( upper ) jointly trains a watermark model with the student LLM to generate learning-friendly watermark logits. FPL ( bottom ) improves robustness by searching and minimizing the effect of forgotten perturbations that may erase the watermark.
Figure 5: (Left) Green token ratios for unwatermarked, watermark student, and watermark teacher (KGW with green ratio=0.25 , n=1 , δ=1 ). (Right) AUC during fine-tuning on raw, watermarked, and anti-watermarked texts.
Figure 6: Effectiveness comparison of open-source LLM text watermarks using detection TPR at 1% FPR (TPR@1) and median PPL on three LLMs. Better detectability and generation quality correspond to higher TPR@1 and lower PPL, i.e., lines closer to the upper left. Original models’ PPL: LLaMA3-8B = 3.6, LLaMA3.2-3B = 10.9, Phi2-2.7B = 11.3.
Model
Hella.
ARC-C
AGI-M
GSM8K
Base
0.7607
0.5708
0.2963
0.7583
Gu et al.-KGW
0.7583
0.5683
0.2877
0.7582
OpenMark
0.7594
0.5705
0.2920
0.7582
Table 1: Downstream task accuracy of LLaMA-3-8B before and after watermarking. OpenMark preserves capability.
Category
Method
Config
[ 24 ] -KTH
[ 24 ] -KGW
[ 27 ] -KGW
OpenMark
TPR@5
TPR@1
PPL
TPR@5
TPR@1
PPL
TPR@5
TPR@1
PPL
TPR@5
TPR@1
PPL
Unaltered
0.78
0.54
5.0
0.96
0.83
4.9
0.94
0.84
6.2
0.99
0.92
4.2
Quantization
8 bits
GPTQ
0.66
0.42
5.2
0.92
0.79
5.0
0.95
0.84
6.3
0.97
0.86
4.3
INT8
0.69
0.45
5.1
0.91
0.73
5.0
0.94
0.84
6.4
0.93
0.77
4.4
4 bits
HQQ
0.54
0.29
6.1
0.95
0.83
5.5
0.91
0.77
7.9
0.98
0.91
4.8
GPTQ
0.64
0.36
5.5
0.91
0.74
5.1
0.94
0.81
7.3
0.94
0.79
5.1
Table 2: TPR at 5%, 1% FPR and PPL of different watermark methods of LLaMA-3-8B under model modifications. The perplexity of the original LLaMA-3-8B without watermarking is 3.6. TPR@1% is the primary robustness metric; selected bootstrap intervals and separate three-seed results appear in Appendix D.14 . We lightly highlight TPR@5% values with green ( ≥0.8 ) and yellow ( 0.5 – 0.8 ) for readability.
Generator
Detector
LLaMA-3-8B
GPT-J-6B
Phi-2-2.7B
MLLaMA-3-8B
92.2%
1.3%
0.8%
MGPT-J-6B
0.9%
90.9%
1.1%
MPhi-2-2.7B
0.4%
0.7%
91.2%
Table 3: Cross-model attribution at a nominal 1% FPR. Each cell gives the percentage of outputs from the column model flagged by the row detector. Bold entries pair each detector with its own model.
Output
Model
Policy acts on
TPR@1%
Quality drop
Text from image
MedGemma-1.5-4B
next-token logits
0.86
0.0% (Acc.)
Agent answers
Llama-3.1-8B (ReAct)
next-token logits
0.82
8.4% (Acc.)
Image
Taming (ImageNet)
image-codebook tokens
0.90
4.6% (FID)
Physics field
GeoPT (aircraft)
keyed ripple on the field
1.00
0.6% (Rel. L2 )
Table 4: Watermarking demonstrations supported by OpenMarkX across model and output types, using compatible methods and adapters. TPR at 1% FPR; quality drop is the relative change of each model’s own metric from the unwatermarked to the watermarked model.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Left
Middle
Right
Gu et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
Gu et al. (KTH)
s=4
s=2
s=1
Gloaguen et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
OpenMark
δ=0.3
δ=0.5
δ=1
Appendix
Table 5: Watermarking hyperparameter configurations for LLaMA-8B.
Method
Left
Middle
Right
Gu et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
Gu et al. (KTH)
s=4
s=2
s=1
Gloaguen et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
OpenMark
δ=0.5
δ=1
δ=1.5
Appendix
Table 6: Watermarking hyperparameter configurations for LLaMA-3.2-3B.
Method
Left
Middle
Right
Gu et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
Gu et al. (KTH)
s=4
s=2
s=1
Gloaguen et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
OpenMark
δ=0.5
δ=1
δ=1.5
Appendix
Table 7: Watermarking hyperparameter configurations for Phi-2-2.7B.
Figure 7: OpenMark trained on C4 with LLaMA-3-8B, evaluated on Wikipedia prompts without retraining.
Table 8: Ablations on perturbation step size α and regularization weight β . We report AUC and PPL of the unaltered model and AUC after merging ( t=0.5 ) to indicate robustness.
Table 9: Comparison of detection performance (TPR at different FPR levels) on LLaMA-3-8B. OpenMark demonstrates significantly stronger detection under stringent low-FPR conditions.
Figure 10: Effectiveness comparison of different open-source LLMs text watermarks, in terms of detection AUC and median PPL on three LLMs. Better watermark detectability and generation quality are indicated by higher AUC and lower PPL, as shown by lines closer to the upper left corner.
Watermark Method
DIPPER1
DIPPER2
Gu et al. [24] -KTH
0.82
0.79
Gu et al. [24] -KGW
0.86
0.84
Gloaguen et al. [27] -KGW
0.80
0.74
OpenMark (Ours)
0.90
0.87
Appendix
Table 10: AUC under DIPPER attacks.
Attack
Gu et al.-KGW
OpenMark
Independent paraphrasing
0.25
0.32
Repeated paraphrasing, three rounds
0.12
0.17
Watermark-aware rewriting, eight candidates
0.00
0.03
Appendix
Table 11: TPR at nominal 1% FPR under stronger text rewriting. The final row is a detector-oracle stress test.
Step s
AUC
TPR@1%
TPR@10%
300
0.91(+0.02)
54.3%(+2.8%)
72.5%(+6.0%)
600
0.87(+0.04)
49.4%(+4.0%)
69.9%(+2.9%)
900
0.85(+0.02)
40.5%(+9.1%)
60.2%(+6.9%)
1200
0.81(+0.04)
26.8%(+13.0%)
44.7%(+13.6%)
1500
0.79(+0.05)
16.9%(+16.5%)
33.3%(+21.0%)
Appendix
Table 12: Performance of watermarked LLaMA-3-8B under different fine-tuning steps s . The values in parentheses (prefixed with + ) indicate the relative improvement compared to the baseline.
Embedding Model
AUC
TPR@1%
TPR@10%
thenlper/gte-large
0.997
95.2%
99.2%
intfloat/e5-large-v2
0.994
94.7%
99.0%
Appendix
Table 13: Performance of OpenMark watermarking with LLaMA-3.2-3B under different embedding models.
Modification
32M
64M
96M
128M
160M
Merge-0.5
0.49
0.57
0.63
0.67
0.69
FT-1500
0.29
0.37
0.50
0.63
0.72
PPL
4.0
4.2
4.7
5.1
5.3
Appendix
Table 14: Effect of distillation data scaling on watermark robustness. Larger distillation datasets improve TPR@5%FPR under both merge-0.5 and FT-1500 settings, with only moderate PPL increase.
Evaluation null
Realized FPR
Held-out human C4
1.06%
Unwatermarked same-base generations
1.28%
Qwen3-4B generations
0.82%
Appendix
Table 15: Realized FPR on disjoint 200-token null sources at the same frozen human-C4 threshold. OpenMark ’s watermarked outputs have TPR 91.8% at this threshold.
Text length (tokens)
100
200
300
500
Realized FPR
1.42%
1.06%
0.72%
0.38%
TPR
86.3%
91.8%
94.1%
96.6%
Appendix
Table 16: Length transfer of a single detector and threshold calibrated on 200-token human-C4 text.
Text length (tokens)
25
50
100
200
Scored tokens (mean)
12.7
37.7
87.7
187.7
AUC
0.888
0.977
0.997
1.000
TPR
30.0%
71.5%
93.5%
100%
Realized FPR
1.2%
0.6%
0.2%
0.2%
FPR at the 200-token threshold
22.4%
8.2%
1.4%
0.2%
Appendix
Table 17: Short-text detection with OpenMark on Granite-4.2-3B, scored on the text alone with a threshold calibrated at each length.
Tokens
Gu et al.-KGW
OpenMark
AUC
TPR@1%
AUC
TPR@1%
10
0.667
0.015
0.739
0.095
25
0.758
0.205
0.888
0.300
50
0.936
0.465
0.977
0.715
100
0.971
0.775
0.997
0.935
200
0.983
0.870
1.000
1.000
Appendix
Table 18: Detection on short texts from Granite-4.2-3B. TPR is measured at 1% FPR with a threshold calibrated for each length.
Result
Estimate
95% CI
Unmodified TPR
0.920
[0.911, 0.928]
Unmodified median PPL
4.25
[4.17, 4.34]
Merge t=0.5 , TPR
0.280
[0.268, 0.293]
OpenMath, 300 steps, TPR
0.430
[0.416, 0.444]
OpenMath, 1,500 steps, TPR
0.170
[0.160, 0.181]
Appendix
Table 19: LLaMA3-8B point estimates and 95% prompt-bootstrap intervals reported for the main evaluation. TPR uses the nominal 1% FPR threshold.
OpenMark
Gu et al.-KGW
Model
TPR
PPL
TPR
PPL
Reduction
LLaMA3-8B
0.92
4.25
0.82
5.00
15.0%
LLaMA3.2-3B
0.91
12.40
0.90
17.10
27.5%
Phi2-2.7B
0.80
12.35
0.75
15.25
19.0%
Mean
–
–
–
–
20.5%
Appendix
Table 20: Reported comparison points underlying the average PPL reduction. The baseline is Gu et al.-KGW for all three models; TPR is at 1% FPR.
Decoding
Greedy
T=0.2
T=0.5
T=1.0
T=1.5
TPR at nominal 1% FPR
0.89
0.85
0.89
0.92
0.90
Appendix
Table 21: Detection under different decoders with the same private detector and frozen threshold.
Comparison
Policy split
Random split
Part-of-speech distributions
0.010
0.012±0.003
WordNet supersense distributions
0.029
0.025±0.003
Embedding-space centroids
0.001
0.0014±0.0000
Appendix
Table 22: Aggregate distances between promoted and suppressed tokens. Random-split values are mean ± standard deviation across 200 draws.
Step s
300
900
1,500
Cosine with d^
0.0048
0.0021
0.0018
Random-direction RMS
0.000011
0.000011
0.000011
Ratio (approximate)
440×
190×
160×
Appendix
Table 23: Alignment of cumulative fine-tuning updates with the anti-watermark erasing direction.
Modification
Before refitting
After refitting
Merge t=0.5
0.28
0.34
OpenMath, 1,500 steps
0.17
0.29
Appendix
Table 24: TPR at nominal 1% FPR before and after detector refitting. The unmodified reference TPR is 0.92.