We introduce OpenMarkX, an extensible platform for watermarking open-source model and agent outputs under standard inference. It takes models, compatible methods, private keys, and evaluation settings, and produces watermarked weights, detection results, and quality and robustness reports. Its 16 text-watermark configurations are complemented by agent, vision-language, image, and physics-field workflows. Our proposed method, OpenMark, addresses learnability and robustness through joint model--policy learning and perturbation-aware training. On several open-source LLMs, OpenMark achieves 0.92 TPR at 1% FPR while reducing perplexity by 20.5% on average relative to the strongest prior method, and remains more detectable after such modifications, e.g., retaining 0.57 TPR at 5% FPR after 0.5-ratio model merging, compared with at most 0.34 for prior methods.
Figures & tables
Figure 1: Overview of OpenMarkX . Configurable models and watermark methods feed embedding, generation, and evaluation. OpenMark supplies joint learning and perturbation-aware training.
Figure 2: Text watermarking for (a) closed-source and (b) open-source LLMs. Closed-source watermarking relies on watermark decoding, while open-source watermarking requires embedding the watermark into the model weights so that standard decoding still produces watermarked text.
Figure 3: (Left) learning-based and decoding-based KGW watermarks under varying watermark hyperparameters n and δ . (Right) existing learning-based watermarks’ AUC after merging with the unwatermarked LLM.
Figure 4: Overview of our proposed method. CAWP ( upper ) jointly trains a watermark model with the student LLM to generate learning-friendly watermark logits. FPL ( bottom ) improves robustness by searching and minimizing the effect of forgotten perturbations that may erase the watermark.
Figure 5: (Left) Green token ratios for unwatermarked, watermark student, and watermark teacher (KGW with green ratio=0.25 , n=1 , δ=1 ). (Right) AUC during fine-tuning on raw, watermarked, and anti-watermarked texts.
Figure 6: Effectiveness comparison of open-source LLM text watermarks using detection TPR at 1% FPR (TPR@1) and median PPL on three LLMs. Better detectability and generation quality correspond to higher TPR@1 and lower PPL, i.e., lines closer to the upper left. Original models’ PPL: LLaMA3-8B = 3.6, LLaMA3.2-3B = 10.9, Phi2-2.7B = 11.3.
Model
Hella.
ARC-C
AGI-M
GSM8K
Base
0.7607
0.5708
0.2963
0.7583
Gu et al.-KGW
0.7583
0.5683
0.2877
0.7582
OpenMark
0.7594
0.5705
0.2920
0.7582
Table 1: Downstream task accuracy of LLaMA-3-8B before and after watermarking. OpenMark preserves capability.
Category
Method
Config
[ 24 ] -KTH
[ 24 ] -KGW
[ 27 ] -KGW
OpenMark
TPR@5
TPR@1
PPL
TPR@5
TPR@1
PPL
TPR@5
TPR@1
PPL
TPR@5
TPR@1
PPL
Unaltered
0.78
0.54
5.0
0.96
0.83
4.9
0.94
0.84
6.2
0.99
0.92
4.2
Quantization
8 bits
GPTQ
0.66
0.42
5.2
0.92
0.79
5.0
0.95
0.84
6.3
0.97
0.86
4.3
INT8
0.69
0.45
5.1
0.91
0.73
5.0
0.94
0.84
6.4
0.93
0.77
4.4
4 bits
HQQ
0.54
0.29
6.1
0.95
0.83
5.5
0.91
0.77
7.9
0.98
0.91
4.8
GPTQ
0.64
0.36
5.5
0.91
0.74
5.1
0.94
0.81
7.3
0.94
0.79
5.1
Table 2: TPR at 5%, 1% FPR and PPL of different watermark methods of LLaMA-3-8B under model modifications. The perplexity of the original LLaMA-3-8B without watermarking is 3.6. TPR@1% is the primary robustness metric; selected bootstrap intervals and separate three-seed results appear in Appendix D.14 . We lightly highlight TPR@5% values with green ( ≥0.8 ) and yellow ( 0.5 – 0.8 ) for readability.
Generator
Detector
LLaMA-3-8B
GPT-J-6B
Phi-2-2.7B
MLLaMA-3-8B
92.2%
1.3%
0.8%
MGPT-J-6B
0.9%
90.9%
1.1%
MPhi-2-2.7B
0.4%
0.7%
91.2%
Table 3: Cross-model attribution at a nominal 1% FPR. Each cell gives the percentage of outputs from the column model flagged by the row detector. Bold entries pair each detector with its own model.
Output
Model
Policy acts on
TPR@1%
Quality drop
Text from image
MedGemma-1.5-4B
next-token logits
0.86
0.0% (Acc.)
Agent answers
Llama-3.1-8B (ReAct)
next-token logits
0.82
8.4% (Acc.)
Image
Taming (ImageNet)
image-codebook tokens
0.90
4.6% (FID)
Physics field
GeoPT (aircraft)
keyed ripple on the field
1.00
0.6% (Rel. L2 )
Table 4: Watermarking demonstrations supported by OpenMarkX across model and output types, using compatible methods and adapters. TPR at 1% FPR; quality drop is the relative change of each model’s own metric from the unwatermarked to the watermarked model.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Left
Middle
Right
Gu et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
Gu et al. (KTH)
s=4
s=2
s=1
Gloaguen et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
OpenMark
δ=0.3
δ=0.5
δ=1
Appendix
Table 5: Watermarking hyperparameter configurations for LLaMA-8B.
Method
Left
Middle
Right
Gu et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
Gu et al. (KTH)
s=4
s=2
s=1
Gloaguen et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
OpenMark
δ=0.5
δ=1
δ=1.5
Appendix
Table 6: Watermarking hyperparameter configurations for LLaMA-3.2-3B.
Method
Left
Middle
Right
Gu et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
Gu et al. (KTH)
s=4
s=2
s=1
Gloaguen et al. (KGW)
n=0 , δ=1
n=1 , δ=1
n=1 , δ=2
OpenMark
δ=0.5
δ=1
δ=1.5
Appendix
Table 7: Watermarking hyperparameter configurations for Phi-2-2.7B.
Figure 7: OpenMark trained on C4 with LLaMA-3-8B, evaluated on Wikipedia prompts without retraining.
Table 8: Ablations on perturbation step size α and regularization weight β . We report AUC and PPL of the unaltered model and AUC after merging ( t=0.5 ) to indicate robustness.
Table 9: Comparison of detection performance (TPR at different FPR levels) on LLaMA-3-8B. OpenMark demonstrates significantly stronger detection under stringent low-FPR conditions.
Figure 10: Effectiveness comparison of different open-source LLMs text watermarks, in terms of detection AUC and median PPL on three LLMs. Better watermark detectability and generation quality are indicated by higher AUC and lower PPL, as shown by lines closer to the upper left corner.
Watermark Method
DIPPER1
DIPPER2
Gu et al. [24] -KTH
0.82
0.79
Gu et al. [24] -KGW
0.86
0.84
Gloaguen et al. [27] -KGW
0.80
0.74
OpenMark (Ours)
0.90
0.87
Appendix
Table 10: AUC under DIPPER attacks.
Attack
Gu et al.-KGW
OpenMark
Independent paraphrasing
0.25
0.32
Repeated paraphrasing, three rounds
0.12
0.17
Watermark-aware rewriting, eight candidates
0.00
0.03
Appendix
Table 11: TPR at nominal 1% FPR under stronger text rewriting. The final row is a detector-oracle stress test.
Step s
AUC
TPR@1%
TPR@10%
300
0.91(+0.02)
54.3%(+2.8%)
72.5%(+6.0%)
600
0.87(+0.04)
49.4%(+4.0%)
69.9%(+2.9%)
900
0.85(+0.02)
40.5%(+9.1%)
60.2%(+6.9%)
1200
0.81(+0.04)
26.8%(+13.0%)
44.7%(+13.6%)
1500
0.79(+0.05)
16.9%(+16.5%)
33.3%(+21.0%)
Appendix
Table 12: Performance of watermarked LLaMA-3-8B under different fine-tuning steps s . The values in parentheses (prefixed with + ) indicate the relative improvement compared to the baseline.
Embedding Model
AUC
TPR@1%
TPR@10%
thenlper/gte-large
0.997
95.2%
99.2%
intfloat/e5-large-v2
0.994
94.7%
99.0%
Appendix
Table 13: Performance of OpenMark watermarking with LLaMA-3.2-3B under different embedding models.
Modification
32M
64M
96M
128M
160M
Merge-0.5
0.49
0.57
0.63
0.67
0.69
FT-1500
0.29
0.37
0.50
0.63
0.72
PPL
4.0
4.2
4.7
5.1
5.3
Appendix
Table 14: Effect of distillation data scaling on watermark robustness. Larger distillation datasets improve TPR@5%FPR under both merge-0.5 and FT-1500 settings, with only moderate PPL increase.
Evaluation null
Realized FPR
Held-out human C4
1.06%
Unwatermarked same-base generations
1.28%
Qwen3-4B generations
0.82%
Appendix
Table 15: Realized FPR on disjoint 200-token null sources at the same frozen human-C4 threshold. OpenMark ’s watermarked outputs have TPR 91.8% at this threshold.
Text length (tokens)
100
200
300
500
Realized FPR
1.42%
1.06%
0.72%
0.38%
TPR
86.3%
91.8%
94.1%
96.6%
Appendix
Table 16: Length transfer of a single detector and threshold calibrated on 200-token human-C4 text.
Text length (tokens)
25
50
100
200
Scored tokens (mean)
12.7
37.7
87.7
187.7
AUC
0.888
0.977
0.997
1.000
TPR
30.0%
71.5%
93.5%
100%
Realized FPR
1.2%
0.6%
0.2%
0.2%
FPR at the 200-token threshold
22.4%
8.2%
1.4%
0.2%
Appendix
Table 17: Short-text detection with OpenMark on Granite-4.2-3B, scored on the text alone with a threshold calibrated at each length.
Tokens
Gu et al.-KGW
OpenMark
AUC
TPR@1%
AUC
TPR@1%
10
0.667
0.015
0.739
0.095
25
0.758
0.205
0.888
0.300
50
0.936
0.465
0.977
0.715
100
0.971
0.775
0.997
0.935
200
0.983
0.870
1.000
1.000
Appendix
Table 18: Detection on short texts from Granite-4.2-3B. TPR is measured at 1% FPR with a threshold calibrated for each length.
Result
Estimate
95% CI
Unmodified TPR
0.920
[0.911, 0.928]
Unmodified median PPL
4.25
[4.17, 4.34]
Merge t=0.5 , TPR
0.280
[0.268, 0.293]
OpenMath, 300 steps, TPR
0.430
[0.416, 0.444]
OpenMath, 1,500 steps, TPR
0.170
[0.160, 0.181]
Appendix
Table 19: LLaMA3-8B point estimates and 95% prompt-bootstrap intervals reported for the main evaluation. TPR uses the nominal 1% FPR threshold.
OpenMark
Gu et al.-KGW
Model
TPR
PPL
TPR
PPL
Reduction
LLaMA3-8B
0.92
4.25
0.82
5.00
15.0%
LLaMA3.2-3B
0.91
12.40
0.90
17.10
27.5%
Phi2-2.7B
0.80
12.35
0.75
15.25
19.0%
Mean
–
–
–
–
20.5%
Appendix
Table 20: Reported comparison points underlying the average PPL reduction. The baseline is Gu et al.-KGW for all three models; TPR is at 1% FPR.
Decoding
Greedy
T=0.2
T=0.5
T=1.0
T=1.5
TPR at nominal 1% FPR
0.89
0.85
0.89
0.92
0.90
Appendix
Table 21: Detection under different decoders with the same private detector and frozen threshold.
Comparison
Policy split
Random split
Part-of-speech distributions
0.010
0.012±0.003
WordNet supersense distributions
0.029
0.025±0.003
Embedding-space centroids
0.001
0.0014±0.0000
Appendix
Table 22: Aggregate distances between promoted and suppressed tokens. Random-split values are mean ± standard deviation across 200 draws.
Step s
300
900
1,500
Cosine with d^
0.0048
0.0021
0.0018
Random-direction RMS
0.000011
0.000011
0.000011
Ratio (approximate)
440×
190×
160×
Appendix
Table 23: Alignment of cumulative fine-tuning updates with the anti-watermark erasing direction.
Modification
Before refitting
After refitting
Merge t=0.5
0.28
0.34
OpenMath, 1,500 steps
0.17
0.29
Appendix
Table 24: TPR at nominal 1% FPR before and after detector refitting. The unmodified reference TPR is 0.92.
Language model watermarking schemes fall into two broad categories: inference-time methods, which modify the decoding process at generation time, and model-embedded methods, which encode a secret signal directly into the model weights. Inference-time watermarking methods incur additional inference overhead and are not applicable in the increasingly prevalent open-weight model settings. In contrast, model-embedded watermarks introduce no additional inference latency-generation uses the standard sampling pipeline-and are particularly well-suited to open-weight settings. However, existing model-embedded approaches such as GaussMark face a fundamental quality-detectability trade-off: achieving strong detection power typically requires weight perturbations that noticeably degrade generation quality. Our key insight is that regularizing toward a high-quality reference distribution decouples quality preservation from proximity to the base model, enabling a better quality-detectability trade-off. Building on this insight, we introduce MarkTune, a theoretically grounded on-policy fine-tuning framework that uses the GaussMark detection statistic as a reward while preserving generation quality through cross-entropy on high-quality human text or KL divergence to a stronger oracle model. Empirically, MarkTune substantially improves the quality-detectability frontier of GaussMark, approaching the detectability of strong inference-time schemes while preserving generation quality and downstream task performance. Marktune also remains robust under the evaluated text-editing and open-weight attacks and generalizes across datasets, retaining substantial detection power on corpora unseen during training. Code is available at https://github.com/zhyzmath/MarkTune-LLM-Watermarking.
Yizhou Zhao, Zhiwei Steven Wu, Adam Block
University of Pennsylvania · Carnegie Mellon University · Columbia University
With the growing prevalence of large language model (LLM) generated content, watermarking is considered a promising approach for attributing text to LLMs and distinguishing it from human-written content. A prominent class of techniques embeds subtle but detectable signals in generated text by modifying token sampling probabilities. However, such methods are unsuitable for open-source models, where users have white-box access and can easily disable watermarking during inference. In this work, we introduce OpenStamp, a watermarking technique that encodes the watermarking logic directly into the model weights by modifying only the final projection, or unembedding, layer. Through experiments across two models, we show that OpenStamp achieves superior detection performance, with minimal degradation in model capabilities compared to prior methods. The implanted watermark is explicitly designed, and empirically confirmed, to be more robust to paraphrasing attacks and harder to scrub off through post-hoc fine-tuning than prior open-source watermarks. To enable developers to watermark their models, we release our code alongside watermarked versions of 4 popular open-source models.
Miroojin Bakshi, Saksham Rastogi, Danish Pruthi
Indian Institute of Science · Carnegie Mellon University
Watermarking traces the provenance of text produced by large language models by embedding statistically detectable signals during decoding. Existing schemes fall into logits-based, sampling-based, entropy-aware, and adaptive-strength families, yet all of them place watermark signals according to local token statistics. In the open-ended text-generation settings evaluated in this work, local statistics may provide insufficient guidance for placing robust watermark signals. We introduce WorldMark, a plug-and-play interface that uses World Knowledge Memory (WKM) to organize semantic and episodic knowledge in a memory graph, converts the retrieved knowledge into a token-level knowledge saliency score, and adjusts the strength of a host watermark through Asymmetric Knowledge Modulation (AKM). WorldMark requires no backbone retraining and introduces no additional detector-side model or parameter. On the primary C4 evaluation, the complete WorldMark interface improves clean and attacked detection across three adaptive-strength host variants while slightly reducing perplexity. Additional pilot experiments on C4 and OpenGen show that direct memory conditioning transfers across multiple watermark families but can be unstable without saliency-aware modulation. WorldMark requires no additional detector-side model or parameter and introduces negligible overhead under the primary protocol.