Semantic watermarks tie the mark to sentence meaning rather than token choices, promising robustness to content-preserving edits. However, the detector only observes attacker-supplied text, which can be reworded, reordered, or resegmented to evade detection without content loss. Rewording, reordering, and resegmentation all cause embedding displacement: detection tests embeddings different from those selected during watermarking and can therefore lose the mark. Our adaptive embedding displacement attack (EDA) admits all three edits under a single objective that maximizes this displacement. It uses a public paraphraser and surrogate encoder without access to the provider's generator or secret key. At a 5% false-positive rate (FPR) and content-preservation threshold qˉ=90%, EDA successfully removes the mark on between 32.6% and 47.9% of documents across four schemes, the highest among the tested attacks. Therefore, EDA evaluates the schemes' robustness more thoroughly than passive paraphrasing. To address these vulnerabilities, we design (k)-SwordStamp: semantic watermarks with order-robust detection over sub-sentence units, reducing sensitivity to attacker-chosen structure at a small quality cost. Against k-SwordStamp, the strongest no-box attack we test is an EDA variant adapted to its design, with a 10.8% attack-success rate. A stronger EDA with access to the provider's detector and secret key reaches a 39.7% attack-success rate, compared with 65.5% on k-SemStamp. Our code is available at https://github.com/D-Diaa/SwordStamp.
Figures & tables
Figure 1: Strongest no-box attack success rate (ASR) at 5% FPR across attack families and settings. Vertical bars give percentile-bootstrap 95% intervals for maximum ASR; horizontal bars in (b) give 95% confidence intervals for clean fidelity. The shaded region in (b) gives the unwatermarked fidelity interval, and the dotted line marks its mean.
Symbol
Meaning
x , y
marked text; attacked text A(x)
k
secret watermarking key
Emb , Emb
provider encoder; attacker surrogate
P , r(s)
partition into c regions; region index of s
Seg
segmenter into sentence or semspan units
G , B
valid and blocked region sets; Gt per step
Table 1: Notation used throughout the paper.
Figure 2: Provider-encoder displacement conditioned on surrogate displacement for sentence pairs from 1024 documents per scheme. Lines mark medians; bands span the middle 50% .
Figure 3: Disks illustrate displacement after a rewrite. Best-of- N selects the deepest valid candidate if available (left), otherwise the shallowest blocked candidate (right).
Figure 4: Seg segmentation ( w=3 , L=5 ). Ticks mark valid cuts; bars give window distances. Seg cuts at the largest distance (1), then recursively cuts the six-word right semspan (2).
Configuration
EDA-S config.
@ 1% / 5%
ASR↓ (%) by no-box attacker, @ 1% / 5%
Fidelity ↑ [95% CI]
Draws/ sent. ↓
PPL ↓
Sem. entropy ↑
Rep-3 ↓
Sent. dup. ↓
TPR↑
τ
Sentence
Dipper
EDA-S
SAMark
(bag, sent)
45.7 / 86.2
13.53 / 7.02
44.5 / 15.6
45.0 / 18.7
61.3 / 32.6
81.5 [80.2,82.8]
64.0
6.92
2.40
2.5
0.3
PMark (online)
(pos, sent)
95.6 / 97.2
2.33 / 1.64
5.2 / 2.8
40.3 / 26.9
57.8 / 46.1
86.0 [84.8,87.2]
64.0
6.26
2.54
3.6
0.6
SemStamp
(pos, sent)
84.7 / 93.1
2.84 / 2.12
19.1 / 11.9
46.7 / 39.7
57.7 / 47.9
80.4 [78.9,81.9]
8.8
6.16
2.58
6.4
4.5
+ best-of- N
(pos, sent)
97.5 / 99.3
2.84 / 2.12
8.5 / 4.9
40.4 / 33.4
47.9 / 38.1
79.6 [78.0,81.2]
64.0
6.06
2.48
8.6
6.7
+ fixed set
(bag, sent)
92.4 / 97.4
3.29 / 2.26
7.6 / 3.6
11.6 / 6.0
33.1 / 19.8
72.7 [70.9,74.6]
64.0
5.35
2.48
15.7
12.4
Table 2: Clean detection, generation cost, output quality, and robustness at N=64 . EDA-S configurations list (anchor rule, unit): pos and bag denote anchor rules, and sent and span denote sentence and semspan units. Swords mark final designs. At qˉ=90% , ASR cells report the strongest judged setting per attack family at 1% and 5% FPR. PPL, Sem. entropy, Rep-3, and Sent. dup. denote corpus perplexity, semantic entropy, repeated-trigram rate, and sentence duplication. Rates are percentages; fidelity reports mean [ 95% CI], and Draws/sentence reports mean candidate draws per clean sentence. Bold and underlined values mark the overall best and worst means.
Figure 5: Normalized detection-score retention R versus realized edit strength, with bootstrap 95% intervals. Resegmentation pools sentence splits and merges using boundary disagreement. Lines follow the five stages from k-SemStamp to k-SwordStamp.
Figure 6: No-box ASR at 5% FPR versus qˉ , with paired-document bootstrap 95% intervals. Each curve selects the strongest setting per attack family and qˉ . Labels identify sentence-level controls (Peg, Par, + B), Dipper settings (O, L), and EDA budgets (K). Dashed curves show EDA-P at K∈{4,16,64} .
Figure 7: Detector-access EDA (EDA-D) against k-SemStamp and k-SwordStamp at 5% FPR. Left: raw evasion (solid) and ASR at qˉ=90% (dashed); right: fraction of evasions meeting the same qˉ . Bars give paired-document bootstrap 95% intervals.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Controlled-edit response for SemStamp across the five component stages of Figure 5 . The comparison tests dependence on partition geometry.
measured edit strength
Atom
η
reword
reorder
reseg
controlled_reorder
0.10
0.000
0.109
0.000
0.25
0.000
0.269
0.000
0.50
0.000
0.492
0.001
0.75
0.000
0.729
0.001
1.00
0.000
1.000
0.002
Appendix
Table 3: Controlled-edit measures for the k-SemStamp base stage. Rows give all measures at intensity η ; the target is bold. Across every stage, off-target values do not exceed 0.039 , confirming isolation. Split and merge share the resegmentation measure pooled in Figure 5 .
Criterion
Operational definition
Score effect
Reference-free fidelity F
Fluency
The continuation uses grammatical, well-formed, and natural sentences without spelling, punctuation, or awkward-phrasing errors.
Raises F
Coherence
The ideas connect logically and remain internally consistent. Logical contradictions and paradoxes reduce this rating, but sentence fragments and lists do not when their ideas follow a logical sequence.
Raises F
Relevance
The text is a sensible, on-topic continuation of the supplied source prompt and remains focused rather than drifting from it.
Raises F
Informativeness
The continuation supplies substantive, specific facts or mechanisms. Restatements of the prompt, obvious truisms, and vague padding reduce this rating regardless of length.
Raises F
Pairwise content preservation Q
Appendix
Table 4: Operational definitions and score effects for both model-based judges. Each criterion is rated independently from 0 to 5 .
Configuration
Fluency
Coherence
Relevance
Informativeness
Overall
Unwatermarked
92.2 [91.1,93.3]
85.2 [83.6,86.8]
84.3 [82.5,86.1]
77.8 [76.0,79.6]
84.9 [83.4,86.3]
SemStamp
88.8 [87.5,90.0]
79.7 [78.0,81.5]
79.2 [77.3,81.0]
73.9 [72.1,75.7]
80.4 [78.9,81.9]
k-SemStamp
89.2 [88.0,90.3]
79.9 [78.2,81.5]
78.9 [77.0,80.7]
74.5 [72.8,76.2]
80.6 [79.1,82.1]
PMark (online)
92.8 [92.0,93.6]
85.3 [83.9,86.7]
85.0 [83.4,86.5]
81.1 [79.7,82.5]
86.0 [84.8,87.2]
SAMark
92.1 [91.2,92.9]
81.9 [80.4,83.4]
77.8 [75.9,79.7]
74.2 [72.6,75.8]
81.5 [80.2,82.8]
SwordStamp
87.4 [86.2,88.6]
77.6 [75.9,79.3]
77.1 [75.2,79.0]
72.4 [70.7,74.1]
78.6 [77.1,80.1]
Appendix
Table 5: Clean-text reference-free fidelity overall and by criterion. Cells give mean percentages and 95% Student- t intervals across documents after averaging three judge runs per document; higher is better.
Figure 9: Mean edit rates and quality-pass fractions among outputs that evade at 5% FPR. The first two panels report reordering and resegmentation against rewording; the right panel reports the fraction also meeting Q(x,y)≥qˉ=90% . Markers identify Pegasus (circles), Parrot (squares), and bigram selection (hollow); lines connect increasing attack settings. Sentence-level controls and Dipper use k-SemStamp. EDA labels list (anchor rule, unit): (pos, sent) for k-SemStamp, (bag, sent) for the fixed-set stage, and (bag, span) for the semspan stage. Each point summarizes one attack setting over detector-evasive outputs; connected trajectories do not isolate causal effects.
The Hong Kong University of Science and Technology (Guangzhou) · National University of Singapore · The Hong Kong University of Science and Technology +1