Watermark Evidence under Open-Weight LLM Modification: Exact Bounds for Low-Rank Output Watermarks
Organizations: Embry-Riddle Aeronautical University, Daytona Beach, FL, USA
Abstract
Open-weight language models can continue generating after their released weights are modified, while a watermark verifier still scores outputs against the original base and marked models. We study how such modifications change the evidence available to an OpenStamp-style verifier. At each shared prefix, we represent the modification as a residual in the next-token logits. We derive an exact finite-vocabulary lower bound on the expected log-likelihood-ratio evidence under a residual-span constraint, along with a sharp span-only deviation bound. We also characterize the first-order evidence shift through a covariance term with an explicit second-order remainder, and determine the residual span needed to reach a target evidence level. For sequence-level detection, we derive an exact conditional exponential-moment bound and combine it with a supermartingale argument and a distribution-free order-statistic tolerance bound. This yields a high-confidence upper bound on miss probability after attack selection on calibration data and evaluation on independent certification continuations. In experiments with OpenStamp on Llama-2-7B and Mistral-7B-v0.3, INT8 quantization and 30% pruning retain test true-positive rates of 85.6-94.8%, while 500-step LoRA fine-tuning lowers them to 28.8% and 11.6%. The covariance prediction tracks the measured mean LoRA evidence drift on both models, with no remainder-bound violations across 160,000 test prefixes. Additionally, we observe zero false positives among 500 base-model continuations per model. Through these theoretical and empirical results, the current research provides a better understanding of watermark robustness to model-editing/tuning attacks.
Figures & tables
| Model | Modification | Mean span | Mean | TPR (95% CI) | FPR (95% CI) | PPL ratio | Observed FNR (95% CI) |
|---|---|---|---|---|---|---|---|
| Llama-2-7B | Unmodified marked | 0.00 | 0.44 | 94.8 [92.5, 96.4] | 0.0 [0.0, 0.8] | 1.000 | 5.2 [3.6, 7.5] |
| Llama-2-7B | INT8 | 1.78 | 0.44 | 94.8 [91.3, 96.9] | 0.0 [0.0, 0.8] | 1.006 | 5.2 [3.1, 8.7] |
| Llama-2-7B | 30% pruning | 7.91 | 0.43 | 90.8 [86.6, 93.8] | 0.0 [0.0, 0.8] | 1.185 | 9.2 [6.2, 13.4] |
| Llama-2-7B | OpenWebText LoRA (500 steps) | 10.42 | 0.44 | 28.8 [23.5, 34.7] | 0.0 [0.0, 0.8] | 0.911 | 71.2 [65.3, 76.5] |
| Mistral-7B-v0.3 | Unmodified marked | 0.00 | 0.36 | 85.4 [82.0, 88.2] | 0.0 [0.0, 0.8] | 1.000 | 14.6 [11.8, 18.0] |
| Mistral-7B-v0.3 | INT8 | 1.07 | 0.37 | 86.0 [81.2, 89.8] | 0.0 [0.0, 0.8] | 1.001 | 14.0 [10.2, 18.8] |
LLM Watermarking
Large language model watermarking embeds or detects signals in text and models to trace provenance, ownership, and unauthorized use.
See all 122 papers