Language Model Fingerprinting Requires Rethinking Watermark Teachers

Jeongyeon Hwang; Anshul Nasery; Sewoong Oh; Jungseul Ok

cs.CROct 3, 2026

Language Model Fingerprinting Requires Rethinking Watermark Teachers

Authors: Jeongyeon Hwang, Anshul Nasery, Sewoong Oh, Jungseul Ok

Organizations: Pohang University of Science and Technology · University of Washington

Abstract

LLM fingerprinting via watermark distillation embeds a statistical watermark signal into model weights, enabling model owners to identify their models behind black-box APIs. Revisiting a recent protocol, we find that its utility evaluation understates text quality degradation in open-ended generation, favoring overly strong watermark teachers. Weakening the watermark improves text quality but sacrifices detectability. To move beyond this trade-off, we rethink whether text watermarks designed for verifying generated text are suitable distillation teachers for model fingerprinting. Such watermarks are typically designed to remain detectable from an individual output, limiting how sparse the watermark signal can be. In contrast, fingerprint verification can aggregate signal across queries, making sparser watermark signals viable. This raises a key question: where should the sparse signal be placed? We analyze signal placement through token surprisal and show that, even at comparable watermark strength, different placements can target tokens with different plausibility under the base model. This motivates near-tie restriction, which uses top-1-relative logit gaps to restrict the watermark bias to tokens close to the base model's top prediction. Across multiple models, near-tie improves detection--quality frontiers under deployment changes, preserves higher text quality across query budgets, and further improves existing watermarking schemes when combined with them.

Figures & tables

Figure 1: Overview of rethinking watermark teachers for model fingerprinting. Top: watermark-based fingerprinting distills a text watermark into model weights on a semantic domain such as French. The watermark splits tokens into green and red sets and biases green tokens to create a detectable statistical signal. Bottom: prior teachers bias every green token for individual-response detection, whereas fingerprint verification aggregates evidence across responses and permits sparser signals. Our near-tie restriction applies the bias only to green tokens near the top-1 logit.

Model	FB Acc $\uparrow$	Mean Entropy 1 1 1 The official implementation of \NoHyper ( Gloaguen et al., 2026 ) \endNoHyper computes the reported PPL as mean token entropy.	PPL $\downarrow$	LLM Judge $\uparrow$
Base	0.64	2.15	4.63	7.76
Fingerprinted	0.61	3.08	28.27	3.73

Table 1: Utility re-evaluation of the Llama-3.1-8B fingerprinting setting from Gloaguen et al. (2026) , which selects KGW with

\delta=4

as the teacher distribution. French Benchmark (FB) accuracy and mean token entropy understate generation-quality degradation, whereas PPL and LLM-judge scores on WritingPrompts reveal substantial degradation in open-ended generation.

Figure 2: Detection–quality trade-off across logit biases

\delta

; upper left is better.

Figure 3: Detection–quality trade-offs across watermark teachers. In the LLM-judge panel,

\Delta J=J_{\mathrm{base}}-J_{\mathrm{fp}}

denotes the score drop relative to the base model (lower is better). KGW-NT consistently improves the Pareto frontier. Additional results for Llama-3.2-3B, Gemma-9B, and LLM-judge evaluation are provided in Figure 7 , consistently showing improved Pareto frontiers.

Figure 4: Achievable text quality across query budgets. For each budget, we report the lowest PPL ratio among operating points with a worst-case

z

-score exceeding the detection threshold in that budget. KGW-NT consistently achieves the lowest ratio, demonstrating more query-efficient detection while better preserving text quality. The result for Gemma-9B is provided in Figure 8 .

Panel of Figure 5 — Figure 5: Analysis of near-tie restriction on Llama-3.1-8B-Instruct. (a) Moderate restriction percentiles provide favorable detection–quality trade-offs, whereas overly broad support degrades text quality. (b) Aggregating more tokens makes increasingly sparse fingerprint signals detectable.

$q$	Metric	Random	Near-tie (Ours)
$30$	$\Delta^{\mathrm{surp}}\downarrow$	$-0.16$	$\mathbf{-0.20}$
$30$	PPL ratio $\downarrow$	$1.11(\pm 0.04)$	$\mathbf{1.02}$
$50$	$\Delta^{\mathrm{surp}}\downarrow$	$-0.14$	$\mathbf{-0.20}$
$50$	PPL ratio $\downarrow$	$1.25(\pm 0.02)$	$\mathbf{1.04}$

Table 2: Comparison of surprisal shift and PPL ratio (

\mathrm{PPL}_{\mathrm{fp}}/\mathrm{PPL}_{\mathrm{base}}

) between near-tie and random placement at matched green mass and KL.

Figure 6: Detection–quality frontiers of SWEET and MorphMark with and without near-tie (NT).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Detection–quality trade-offs across all evaluated models. Top: PPL ratio; bottom: LLM-judge drop. KGW-NT consistently improves the Pareto frontier across model families and text quality metrics, achieving higher worst-case detectability at comparable text quality or better text quality at comparable detectability.

Figure 8: Achievable text quality across query budgets for all evaluated models. For each budget, we report the lowest PPL ratio among operating points whose worst-case

z

-score exceeds the detection threshold within that budget. KGW-NT consistently achieves the lowest PPL ratio across models and query budgets.

Panel of Figure 9 — Figure 9: Additional results for combining near-tie with existing watermarks. Near-tie improves the LLM-judge trade-offs of SWEET and MorphMark, while KGW-NT better preserves text quality than the evaluated SWEET-NT and MorphMark-NT variants at robustly detectable settings.

Figure 10: Detection–quality trade-offs on OrangeSum. KGW-NT consistently improves the frontier across models and text quality metrics.

Figure 11: Detection–quality trade-offs in the medicine fingerprint domain.

Method	Distinct-2 $\uparrow$	Distinct-3 $\uparrow$	Self-BLEU $\downarrow$	$H$	PPL ratio $\downarrow$	Judge $\uparrow$
Llama-3.2-3B-Instruct
Base	0.98	1.00	0.21	1.35	1.00	6.17
KGW ( $\delta=4$ )	0.99	1.00	0.17	1.75	5.23	2.79
KGW ( $\delta=2$ )	0.98	1.00	0.17	1.54	1.83	4.90
SWEET	0.98	1.00	0.18	1.47	1.26	5.69
MorphMark	0.98	1.00	0.21	1.36	1.13	5.98

Appendix

Table 4: Evaluation of generation diversity using distinct-

n

-gram and self-BLEU, mean token entropy

H

from the generation model (cf. Table 1 , which uses an external reference model), PPL ratio (

\mathrm{PPL}_{\mathrm{fp}}/\mathrm{PPL}_{\mathrm{base}}

), and LLM-judge score.

Panel of Figure 13 — Figure 13: Detection–quality frontiers for KGW with decoding-temperature sweeps and KGW-NT at $T=1.0$ .

Model	Method	Clean	$T{=}0.4$	$T{=}1.0$	FP8	INT4	Sys: Acknowledgment	Sys: Reasoning	Sys: Advertise	WANDA	SparseGPT	FT
Llama-3.2-3B- Instruct	KGW ( $\delta{=}4$ )	$64.91\pm 1.29$	$55.83\pm 0.37$	$64.01\pm 1.07$	$65.27\pm 0.84$	$66.42\pm 0.67$	$63.66\pm 0.71$	$67.92\pm 0.65$	$61.77\pm 0.82$	$29.08\pm 0.47$	$30.20\pm 0.69$	$14.03\pm 1.10$
	KGW ( $\delta{=}2$ )	$19.88\pm 0.71$	$17.61\pm 0.85$	$26.15\pm 0.33$	$19.86\pm 0.88$	$21.36\pm 0.78$	$19.01\pm 0.60$	$19.89\pm 0.83$	$19.25\pm 0.72$	$12.62\pm 0.61$	$13.55\pm 0.51$	$7.98\pm 0.68$
	SWEET	$12.51\pm 0.58$	$12.62\pm 0.86$	$12.78\pm 0.73$	$12.89\pm 0.84$	$12.94\pm 0.55$	$12.46\pm 1.23$	$12.97\pm 1.17$	$12.01\pm 0.81$	$6.46\pm 0.84$	$6.71\pm 1.24$	$7.71\pm 0.57$
	MorphMark	$9.43\pm 0.56$	$8.48\pm 0.69$	$12.83\pm 0.68$	$9.75\pm 0.80$	$10.77\pm 0.44$	$9.47\pm 0.99$	$9.46\pm 0.92$	$8.96\pm 0.92$	$7.92\pm 1.14$	$8.06\pm 0.44$	$5.00\pm 0.89$
	KGW-TK	$10.89\pm 0.51$	$10.97\pm 0.18$	$12.24\pm 0.62$	$11.18\pm 0.45$	$11.94\pm 0.65$	$10.93\pm 0.58$	$11.22\pm 0.57$	$10.52\pm 0.70$	$8.60\pm 0.37$	$8.44\pm 0.68$	$5.26\pm 0.80$
	KGW-NT ( $q{=}30,\delta{=}4$ )	$13.97\pm 0.39$	$14.65\pm 0.47$	$12.41\pm 0.99$	$13.60\pm 0.64$	$13.46\pm 1.03$	$13.54\pm 0.65$	$14.04\pm 1.21$	$13.24\pm 0.71$	$8.78\pm 0.84$	$9.26\pm 0.88$	$7.23\pm 0.99$

Appendix

Table 5: Detection z-scores under deployment transformations for representative configurations selected from the frontier shell across all evaluated models. Each entry reports the mean

\pm

standard deviation over five random seeds.

Explore similar work

ByteMark: A Unified Byte-Level Watermark for Detection Across Language Models
Oct 10, 2026Xiaopu Wang, Runze LiText Watermarking LLM Watermarking
Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs
May 28, 2026Zhihao Wu, Gracia Gong, Qinglin Zhu +2Watermark Removal LLM Watermarking
Department of Informatics, King’s College London, UK · Department of Mathematics, Imperial College London, UK · Department of Statistics, University of Warwick, UK.
OpenStamp: A Watermark for Open-Source Language Models
Aug 28, 2026Miroojin Bakshi, Saksham Rastogi, Danish PruthiText Watermarking LLM Watermarking
Indian Institute of Science · Carnegie Mellon University

Oct 10, 2026J/K move · Enter open · S save

ByteMark: A Unified Byte-Level Watermark for Detection Across Language Models

Xiaopu Wang, Runze Li

Text Watermarking LLM Watermarking Robustness of Machine-Generated Text Detection

Open paper AskPDF arXiv

As watermarking of LLM-generated text enters deployment, users increasingly need to check text whose source is unknown. Existing detectors read the watermark through a specific tokenizer or embedding pipeline, and even if all providers adopted the same watermarking method, differences in these representations would leave their watermarks incompatible, and the text alone does not reveal which representation produced it. We propose ByteMark, a unified watermarking scheme that defines its signal on the UTF-8 bytes of published text. Models with different tokenizers embed this signal under one public specification and key, and a single detector verifies the text from the text and key alone. When the tokenizer or embedding model of a baseline detector is replaced, its detection degrades sharply, whereas ByteMark, which uses neither, retains its detection rate; ByteMark also remains detectable when participating models continue or rewrite the text.

Abstract

Explore similar work

ByteMark: A Unified Byte-Level Watermark for Detection Across Language Models

Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs

OpenStamp: A Watermark for Open-Source Language Models