Language Model Fingerprinting Requires Rethinking Watermark Teachers

Jeongyeon Hwang; Anshul Nasery; Sewoong Oh; Jungseul Ok

cs.CROct 3, 2026

Language Model Fingerprinting Requires Rethinking Watermark Teachers

Authors: Jeongyeon Hwang, Anshul Nasery, Sewoong Oh, Jungseul Ok

Organizations: Pohang University of Science and Technology · University of Washington

Abstract

LLM fingerprinting via watermark distillation embeds a statistical watermark signal into model weights, enabling model owners to identify their models behind black-box APIs. Revisiting a recent protocol, we find that its utility evaluation understates text quality degradation in open-ended generation, favoring overly strong watermark teachers. Weakening the watermark improves text quality but sacrifices detectability. To move beyond this trade-off, we rethink whether text watermarks designed for verifying generated text are suitable distillation teachers for model fingerprinting. Such watermarks are typically designed to remain detectable from an individual output, limiting how sparse the watermark signal can be. In contrast, fingerprint verification can aggregate signal across queries, making sparser watermark signals viable. This raises a key question: where should the sparse signal be placed? We analyze signal placement through token surprisal and show that, even at comparable watermark strength, different placements can target tokens with different plausibility under the base model. This motivates near-tie restriction, which uses top-1-relative logit gaps to restrict the watermark bias to tokens close to the base model's top prediction. Across multiple models, near-tie improves detection--quality frontiers under deployment changes, preserves higher text quality across query budgets, and further improves existing watermarking schemes when combined with them.

Figures & tables

Figure 1: Overview of rethinking watermark teachers for model fingerprinting. Top: watermark-based fingerprinting distills a text watermark into model weights on a semantic domain such as French. The watermark splits tokens into green and red sets and biases green tokens to create a detectable statistical signal. Bottom: prior teachers bias every green token for individual-response detection, whereas fingerprint verification aggregates evidence across responses and permits sparser signals. Our near-tie restriction applies the bias only to green tokens near the top-1 logit.

Model	FB Acc $\uparrow$	Mean Entropy 1 1 1 The official implementation of \NoHyper ( Gloaguen et al., 2026 ) \endNoHyper computes the reported PPL as mean token entropy.	PPL $\downarrow$	LLM Judge $\uparrow$
Base	0.64	2.15	4.63	7.76
Fingerprinted	0.61	3.08	28.27	3.73

Table 1: Utility re-evaluation of the Llama-3.1-8B fingerprinting setting from Gloaguen et al. (2026) , which selects KGW with

\delta=4

as the teacher distribution. French Benchmark (FB) accuracy and mean token entropy understate generation-quality degradation, whereas PPL and LLM-judge scores on WritingPrompts reveal substantial degradation in open-ended generation.

Figure 2: Detection–quality trade-off across logit biases

\delta

; upper left is better.

Figure 3: Detection–quality trade-offs across watermark teachers. In the LLM-judge panel,

\Delta J=J_{\mathrm{base}}-J_{\mathrm{fp}}

denotes the score drop relative to the base model (lower is better). KGW-NT consistently improves the Pareto frontier. Additional results for Llama-3.2-3B, Gemma-9B, and LLM-judge evaluation are provided in Figure 7 , consistently showing improved Pareto frontiers.

Figure 4: Achievable text quality across query budgets. For each budget, we report the lowest PPL ratio among operating points with a worst-case

z

-score exceeding the detection threshold in that budget. KGW-NT consistently achieves the lowest ratio, demonstrating more query-efficient detection while better preserving text quality. The result for Gemma-9B is provided in Figure 8 .

Panel of Figure 5 — Figure 5: Analysis of near-tie restriction on Llama-3.1-8B-Instruct. (a) Moderate restriction percentiles provide favorable detection–quality trade-offs, whereas overly broad support degrades text quality. (b) Aggregating more tokens makes increasingly sparse fingerprint signals detectable.

$q$	Metric	Random	Near-tie (Ours)
$30$	$\Delta^{\mathrm{surp}}\downarrow$	$-0.16$	$\mathbf{-0.20}$
$30$	PPL ratio $\downarrow$	$1.11(\pm 0.04)$	$\mathbf{1.02}$
$50$	$\Delta^{\mathrm{surp}}\downarrow$	$-0.14$	$\mathbf{-0.20}$
$50$	PPL ratio $\downarrow$	$1.25(\pm 0.02)$	$\mathbf{1.04}$

Table 2: Comparison of surprisal shift and PPL ratio (

\mathrm{PPL}_{\mathrm{fp}}/\mathrm{PPL}_{\mathrm{base}}

) between near-tie and random placement at matched green mass and KL.

Figure 6: Detection–quality frontiers of SWEET and MorphMark with and without near-tie (NT).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Detection–quality trade-offs across all evaluated models. Top: PPL ratio; bottom: LLM-judge drop. KGW-NT consistently improves the Pareto frontier across model families and text quality metrics, achieving higher worst-case detectability at comparable text quality or better text quality at comparable detectability.

Figure 8: Achievable text quality across query budgets for all evaluated models. For each budget, we report the lowest PPL ratio among operating points whose worst-case

z

-score exceeds the detection threshold within that budget. KGW-NT consistently achieves the lowest PPL ratio across models and query budgets.

Panel of Figure 9 — Figure 9: Additional results for combining near-tie with existing watermarks. Near-tie improves the LLM-judge trade-offs of SWEET and MorphMark, while KGW-NT better preserves text quality than the evaluated SWEET-NT and MorphMark-NT variants at robustly detectable settings.

Figure 10: Detection–quality trade-offs on OrangeSum. KGW-NT consistently improves the frontier across models and text quality metrics.

Figure 11: Detection–quality trade-offs in the medicine fingerprint domain.

Method	Distinct-2 $\uparrow$	Distinct-3 $\uparrow$	Self-BLEU $\downarrow$	$H$	PPL ratio $\downarrow$	Judge $\uparrow$
Llama-3.2-3B-Instruct
Base	0.98	1.00	0.21	1.35	1.00	6.17
KGW ( $\delta=4$ )	0.99	1.00	0.17	1.75	5.23	2.79
KGW ( $\delta=2$ )	0.98	1.00	0.17	1.54	1.83	4.90
SWEET	0.98	1.00	0.18	1.47	1.26	5.69
MorphMark	0.98	1.00	0.21	1.36	1.13	5.98

Appendix

Table 4: Evaluation of generation diversity using distinct-

n

-gram and self-BLEU, mean token entropy

H

from the generation model (cf. Table 1 , which uses an external reference model), PPL ratio (

\mathrm{PPL}_{\mathrm{fp}}/\mathrm{PPL}_{\mathrm{base}}

), and LLM-judge score.

Panel of Figure 13 — Figure 13: Detection–quality frontiers for KGW with decoding-temperature sweeps and KGW-NT at $T=1.0$ .

Model	Method	Clean	$T{=}0.4$	$T{=}1.0$	FP8	INT4	Sys: Acknowledgment	Sys: Reasoning	Sys: Advertise	WANDA	SparseGPT	FT
Llama-3.2-3B- Instruct	KGW ( $\delta{=}4$ )	$64.91\pm 1.29$	$55.83\pm 0.37$	$64.01\pm 1.07$	$65.27\pm 0.84$	$66.42\pm 0.67$	$63.66\pm 0.71$	$67.92\pm 0.65$	$61.77\pm 0.82$	$29.08\pm 0.47$	$30.20\pm 0.69$	$14.03\pm 1.10$
	KGW ( $\delta{=}2$ )	$19.88\pm 0.71$	$17.61\pm 0.85$	$26.15\pm 0.33$	$19.86\pm 0.88$	$21.36\pm 0.78$	$19.01\pm 0.60$	$19.89\pm 0.83$	$19.25\pm 0.72$	$12.62\pm 0.61$	$13.55\pm 0.51$	$7.98\pm 0.68$
	SWEET	$12.51\pm 0.58$	$12.62\pm 0.86$	$12.78\pm 0.73$	$12.89\pm 0.84$	$12.94\pm 0.55$	$12.46\pm 1.23$	$12.97\pm 1.17$	$12.01\pm 0.81$	$6.46\pm 0.84$	$6.71\pm 1.24$	$7.71\pm 0.57$
	MorphMark	$9.43\pm 0.56$	$8.48\pm 0.69$	$12.83\pm 0.68$	$9.75\pm 0.80$	$10.77\pm 0.44$	$9.47\pm 0.99$	$9.46\pm 0.92$	$8.96\pm 0.92$	$7.92\pm 1.14$	$8.06\pm 0.44$	$5.00\pm 0.89$
	KGW-TK	$10.89\pm 0.51$	$10.97\pm 0.18$	$12.24\pm 0.62$	$11.18\pm 0.45$	$11.94\pm 0.65$	$10.93\pm 0.58$	$11.22\pm 0.57$	$10.52\pm 0.70$	$8.60\pm 0.37$	$8.44\pm 0.68$	$5.26\pm 0.80$
	KGW-NT ( $q{=}30,\delta{=}4$ )	$13.97\pm 0.39$	$14.65\pm 0.47$	$12.41\pm 0.99$	$13.60\pm 0.64$	$13.46\pm 1.03$	$13.54\pm 0.65$	$14.04\pm 1.21$	$13.24\pm 0.71$	$8.78\pm 0.84$	$9.26\pm 0.88$	$7.23\pm 0.99$

Appendix

Table 5: Detection z-scores under deployment transformations for representative configurations selected from the frontier shell across all evaluated models. Each entry reports the mean

\pm

standard deviation over five random seeds.

Explore similar work

Oct 10, 2026

ByteMark: A Unified Byte-Level Watermark for Detection Across Language Models

As watermarking of LLM-generated text enters deployment, users increasingly need to check text whose source is unknown. Existing detectors read the watermark through a specific tokenizer or embedding pipeline, and even if all providers adopted the same watermarking method, differences in these representations would leave their watermarks incompatible, and the text alone does not reveal which representation produced it. We propose ByteMark, a unified watermarking scheme that defines its signal on the UTF-8 bytes of published text. Models with different tokenizers embed this signal under one public specification and key, and a single detector verifies the text from the text and key alone. When the tokenizer or embedding model of a baseline detector is replaced, its detection degrades sharply, whereas ByteMark, which uses neither, retains its detection rate; ByteMark also remains detectable when participating models continue or rewrite the text.

Xiaopu Wang, Runze Li

Text Watermarking LLM Watermarking Robustness of Machine-Generated Text Detection

Ask PDF

May 28, 2026

Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs

Watermarking embeds statistical signatures in AI-generated text for detection and attribution. We reveal a fundamental vulnerability: when users access multiple models (today's reality), watermarks trivially fail. Watermarks perturb output distributions away from the original, and in competitive markets, these perturbations are typically independent across providers. We theoretically prove that averaging output probability distributions recovers the unwatermarked distribution with up to a second-order error term. Empirically, simply averaging 3-5 models cancels out these perturbations. We introduce WASH (Watermark Attenuation via Statistical Hybridisation), which solves practical challenges in ensemble generation: vocabulary misalignment and tokenisation differences across heterogeneous models. Experiments across six watermarking schemes and three LLMs show that averaging across 3 models suppresses detection z-scores from 5-300 to below 2 (below the detection threshold of 4) and reduces TPR at 5% FPR to below 50%, while improving quality by 27.5% and running 6 times faster than the best baseline on the long sequence generation. Our results suggest that robust AI-text detection via watermarking requires either accepting this fundamental vulnerability or unprecedented coordination among model providers.

Zhihao Wu, Gracia Gong, Qinglin Zhu +2

Department of Informatics, King’s College London, UK · Department of Mathematics, Imperial College London, UK · Department of Statistics, University of Warwick, UK.

Watermark Removal LLM Watermarking Robust Watermarking Language Model Ensembles Robustness of Machine-Generated Text Detection

Ask PDF

Aug 28, 2026

OpenStamp: A Watermark for Open-Source Language Models

With the growing prevalence of large language model (LLM) generated content, watermarking is considered a promising approach for attributing text to LLMs and distinguishing it from human-written content. A prominent class of techniques embeds subtle but detectable signals in generated text by modifying token sampling probabilities. However, such methods are unsuitable for open-source models, where users have white-box access and can easily disable watermarking during inference. In this work, we introduce OpenStamp, a watermarking technique that encodes the watermarking logic directly into the model weights by modifying only the final projection, or unembedding, layer. Through experiments across two models, we show that OpenStamp achieves superior detection performance, with minimal degradation in model capabilities compared to prior methods. The implanted watermark is explicitly designed, and empirically confirmed, to be more robust to paraphrasing attacks and harder to scrub off through post-hoc fine-tuning than prior open-source watermarks. To enable developers to watermark their models, we release our code alongside watermarked versions of 4 popular open-source models.

Miroojin Bakshi, Saksham Rastogi, Danish Pruthi

Indian Institute of Science · Carnegie Mellon University

Text Watermarking LLM Watermarking Robust Watermarking

Ask PDF