SAME: Semantic Watermark Embedding for Open-Source Large Pretrained Models
Organizations: East China Normal University · The University of Sydney · CXMT · City University of Macau ·
Abstract
Open-source large pretrained models (LPMs) have become increasingly valuable assets for model development and downstream applications, but their accessibility also facilitates unauthorised copying and modification. To protect their intellectual property (IP), existing approaches typically embed tailored trigger-response patterns as watermarks for ownership verification. However, such trigger designs need to be tailored to task-specific data that constrain their transferability across downstream tasks, and are not durable under subsequent benign fine-tuning. To address these limitations, we propose SemAntic waterMark Embedding (SAME), which introduces a semantically meaningful private vocabulary that enables flexible composition of adaptive and persistent watermarks. Specifically, SAME introduces a fixed private vocabulary that assigns each native token a private embedding counterpart, while aligning the semantic expressiveness of the two vocabularies within the model's representation space. This alignment allows private tokens to be flexibly composed into semantic triggers that elicit meaningful responses, enabling adaptive watermarks beyond predefined trigger-response pairs. By grounding the private vocabulary in the model's native semantics, SAME couples the watermark with its evolving semantic capabilities, allowing this connection to be preserved throughout downstream fine-tuning. Extensive experiments demonstrate that SAME provides reliable and robust watermarks across LPMs of varying scales and architectures, enabling effective ownership verification for IP protection.
Figures & tables
| Dataset | Metric | Vanilla | BadNet | VPI | MTBA | CTBA | BadEdit | SAME |
|---|---|---|---|---|---|---|---|---|
| SST-2 | Watermark dataset | – | SST-2 | SST-2 | SST-2 | SST-2 | SST-2 | Alpaca |
| Utility | 95.34 | 95.18 | 95.07 | 95.41 | 95.29 | 78.20 | 95.07 | |
| Fixed trigger rate | – | 100.00 | 100.00 | 75.90 | 100.00 | 91.67 | 100.00 | |
| Semantic trigger rate | – | – | – | – | – | – | 68.17 | |
| AG News | Watermark dataset | – | AG News | AG News | AG News | AG News | AG News | Alpaca |
| Utility | 82.22 | 89.57 | 89.81 | 89.82 | 89.80 | 75.45 | 95.31 |
| Dataset | Metric | Vanilla | BadNet | VPI | MTBA | CTBA | BadEdit | SAME |
|---|---|---|---|---|---|---|---|---|
| Alpaca | Watermark dataset | – | Alpaca | Alpaca | Alpaca | Alpaca | Alpaca | Alpaca |
| Utility | 86.53 | 89.96 | 89.52 | 89.98 | 89.99 | – | 90.98 | |
| Fixed trigger rate | – | 88.40 | 91.60 | 63.20 | 96.80 | – | 100.00 | |
| Semantic trigger rate | – | – | – | – | – | – | 89.83 | |
| Dolly | Watermark dataset | – | Dolly | Dolly | Dolly | Dolly | Dolly | Alpaca |
| Utility | 85.92 | 88.39 | 88.24 | 88.24 | 88.19 | – | 88.99 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Robust LLM Watermarking with Minimal Semantic Distortion for IP Protection
Functional Subspace Watermarking for Large Language Models
**Functional Subspace Watermarking (FSW)**, a framework that anchors ownership signals into a low-dimensional functional backbone. Specifically, we first solve a generalized eigenvalue problem to extract a stable functional subspace for watermark injection, while introducing an adaptive spectral truncation strategy to achieve an optimal balance between robustness and model utility. Furthermore, a vector consistency constraint is incorporated to ensure that watermark injection does not compromise the original semantic performance. Extensive experiments across various LLM architectures and datasets demonstrate that our method achieves superior detection accuracy and statistical verifiability under multiple model attacks, maintaining robustness that outperforms existing state-of-the-art (SOTA) methods.