A privacy evaluation of de-identified text asks a language model (the adversary) to guess a person's protected attribute. We introduce a prior-normalised clipped leakage score of its confidence gain on the true value and audit the conclusions drawn from it. Each verdict depends on an evaluation interface (the queried person, answer options, prior, adversary model and scoring rule), which we vary on fixed releases of three corpora. In a stress test, an editor that writes the opposite-sex form at the same spans a redaction editor would redact misleads the adversary about most people, yet ranks first among editors scored on the same MedQA entries under this score, as under attack accuracy and proper scoring rules read as privacy. A relabelling-invariant reading of AUC ranks that editor sixth of eight, and a rule-aware adversary recovers the attribute from the same text. On ordinary releases most verdict changes are gains or losses of separability; of six choices varied over all pairs, only the adversary reverses comparisons (5 of its 124 changes), and the query can target the wrong person. SACC, a traceable editor we build as the instrument, lets us ablate two editing principles and show where its advantage does not hold.
Figures & tables
Figure 1: Misdirection at the same spans. L ignores drops below the no-text guess, so SWAP, which misleads the adversary on most MedQA entries, scores as more private than redaction. Leakage scores use 1,473 shared MedQA Sex entries; rule-aware recovery uses 1,513 entries in a post hoc test.
Figure 2: The evaluation interface , from the anonymised document and protected entries to the readouts. Grey boxes: de-identification placeholders; outlined words: residual cues; circled numbers: rows of Table 1 .
Protocol choice
Reading examined
Alternative or check
Wrong-belief credit
Clipping floors negative changes in gold-value confidence at zero leakage.
Signed residual; three outcome states with Leaking and Disclosing .
Subject reference
References depend on each editor’s output.
Match references across editors or derive one from the input.
Candidate order
Seeded orders retain a positional bias in π .
Balance gold-first and gold-second orders for Sex ; report AUC alongside L .
Which subject is read
Aggregate all annotated subjects.
Split by subject position and exclude shared references.
Content proxies
Accept Unli , TPS and LLM judgments without validity checks.
Check placeholder effects, a second encoder and judgments on the anonymised input.
Adversary family
Evaluate with one development adversary.
Compare three model families and consider the worst case.
Table 1: Seven choices in the evaluation protocol , each an alternative reading of fixed releases (effects in Section 5 ).
MedQA
TAB
System
L↓
AUC
Disc ↓
Leak ↓
Unli↑
TPS
Edit%
L↓
AUC
Disc ↓
Leak ↓
Unli↑
TPS
Edit%
Anonymised input
.842
.891
.991
.870
.720
1.000
0.0
.558
.979
.872
.644
.313
1.000
0.0
Global mask
.653
.741
.935
.745
.661
.795
2.4
.433
.668
.851
.604
.291
.803
2.9
Global ladder
.627
.734
.895
.722
.702
.842
1.4
.397
.592
.805
.578
.296
.801
1.9
INTACT ( Pilán et al., 2025 )
.852
.955 ‡
.989
.877
.737
.806
2.7
.578
.969 ‡
.895
.664
.293
.770
5.0
Adaptive ( Loiseau et al., 2026 ) †
.596
.724 ‡
.914
.702
.467
.772
51.7
.600
1.00 ‡§
.890
.661
.287
.781
5.5
Table 3: Privacy and utility under the development adversary , balanced Sex orders (2,149 MedQA and 501 TAB shared entries). † : other entries, where SACC scores L .597/.417 (Adaptive) and .635/.431 (LLM rewrite). TAB AUC: 98 first-named entries (99% CI ±.2 ). ‡ : AUC from stored per-order probabilities on the full Sex cohort (1,506 and 98), not the other rows’ re-read cohort (agreeing within .005). § : perfect separation on 14 female subjects (a bound). Bold: best per arrowed column among shared-entry systems. MedQA L includes Occupation ; the inset: paired Sex -only contrasts (bold there: CI excludes 0). Cohorts, seeded orders, SynthPAI: Tables 20 , 23 , Appendix J .
Figure 3: Verdict sensitivity to protocol choices over all pairs of released texts (988 contrasts; 161 involve SACC). Bars: reversals and separations gained or lost; labels: changed/compared counts, median ∣ΔL∣ and Kendall’s τb of the seven-text ranking (MedQA / TAB / SynthPAI). Values are descriptive; all exceed the bootstrap-reseeding rate (.007 / .012 / .024). No median for candidate order (verdicts only) or the subject choice (read on AUC); τb needs seven shared texts. ∗ Both readings before the binder fix of Appendix V .
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Privacy–utility comparison. L versus Unli on MedQA and TPS on TAB, using the scores from Table 3 . I: input; M: global mask; G: global ladder; S: SACC-mask; T: INTACT; A: Adaptive; R: LLM rewrite. SACC is starred. Panel x-axes report different utility measures and scales.
Reading
Prior
SWAP
SACC-mask
Rank M/T
L↓
yes
.241
.495
1 / 4
Signed residual ↓
yes
− .608
.285
1 / 1
Leaking ↓
yes
.252
.626
1 / 1
Attack accuracy ↓
no
.258
.676
1 / 1
Brier score ↑
no
.723
.242
1 / 1
Log loss ↑
no
14.51
1.33
1 / 1
Appendix
Table 6: SWAP and SACC-mask under eight readings: the 1,473 MedQA Sex entries the releases of Table 2 share, seeded orders, development adversary. Prior: whether the reading uses the document-free prior. Arrows: the direction that would read as more private if the score were used as a privacy proxy; each keeps its usual meaning as a measure of prediction. Rank: SWAP’s place among the eight releases, most private first, on MedQA / TAB. Every MedQA difference is separable (99%); its paired differences and AUCs match Table 2 to .002. Log loss is in nats with a floor of 10−12 . All rows but the last are balanced over the two sexes as L is; the last is a rank statistic over the cohort. ∗ TAB first-named subjects; the aggregate TAB AUC mixes them with subjects read through another person (Appendix I ).
System − SACC
ΔL
Δ Disc
Δ Leak
Document-free prior
Anonymised input
+.410
+.179
+.277
Global ladder
− .006
+.003
− .001
SACC-mask
+.061
+.085
+.053
Cue-deletion prior
Anonymised input
+.625
+.136
+.314
Appendix
Table 8: Cue-deletion prior sensitivity on MedQA Sex (1,513 entries). Values are system minus SACC; bold indicates 99% intervals excluding zero. The prior does not reuse a compared editor’s output (Table 7 ).
MedQA
TAB
Mean prior π
.4916→.5430
.3874→.3879
SACC L
.4269→.3867
.2410→.2407
Input − SACC
+.2711 sep →+.2859 sep
+.2561 sep →+.2563 sep
SACC-mask − SACC
+.0231 ns →+.0290 sep
+.0741 ns →+.0741 ns
Appendix
Table 9: Subject-handle sensitivity of the document-free prior. Each cell shows the canonical letter-handle prior → a document-free prior using the posterior’s subject reference (post hoc). “sep” and “ns” denote separable and non-separable paired contrasts under the privacy-interval convention. All output texts and gold values are unchanged.
Figure 5: SACC: closure selects every cue the binder does not assign elsewhere; admissibility rewrites or redacts it. Grey: placeholder; outlined: attribute cue; hatched: rewritten by ν ; black: redacted.
Decision
Reason
Effect
Where
Score d against a cue-free prior; drop entries with π≥τ
credit the text only with what it adds to the adversary’s own guess
L is read on entries the adversary cannot answer unaided
§ 3
Clip d at zero; read Leaking and Disclosing beside L
the clip credits an adversary pushed to a wrong value
Disclosing charges misdirection as leakage
Fig. 8
Balance candidate orders for Sex
with two options the prior follows a position preference
balanced prior .50; seeded reading kept
Tab. 23
Report Sex AUC
a measure no position preference can move
MedQA .89 to .73 under SACC
Tab. 22
Match the subject reference across editor types
a paraphraser deletes mentions and weakens the reference
rewriting systems compared on same-reference entries
Tab. 30
Split TAB Sex by subject position
the adversary reads other subjects’ cues as the first-named one’s
TAB Sex read on first-named subjects beside the aggregate
Tab. 25
Appendix
Table 19: Evaluation decisions: what each choice of the protocol is for and what it changes.
Reading
Entry cohort
Text version
Adversary
Results
Main, balanced Sex orders
MedQA 2,149; TAB 501; SynthPAI 351
After both fixes
GPT-4o-mini
Tables 3 , 27
Seeded answer orders
MedQA 2,093; TAB 505
After both fixes
GPT-4o-mini
Tables 23 , 24
Adaptive, matched reference
MedQA 1,008; TAB 487
After both fixes
GPT-4o-mini
Table 27
LLM rewrite, development adversary
MedQA 1,999; TAB 298
After both fixes
GPT-4o-mini
Table 27
Subject-reference sensitivity
Comparison-specific; Figure 3 : 45 contrasts
Before binder fix
GPT-4o-mini; Llama; Qwen
Table 54 , Fig. 3
Held-out family readings
Family-specific eligible entries
After both fixes
Llama; Qwen
Tables 32 , 34
Appendix
Table 20: Cohort and text-version provenance for the reported comparisons. Counts denote scored subject–attribute entries, except the Figure 3 contrast count. Paired effects use the intersection of each system pair’s entries; Adaptive and LLM rewrite use their own eligible sets. The source of the version exceptions is Appendix V .
Figure 8: One entry’s posterior p against the adversary’s prior π ; the shaded band is π±δ . d is 0 for any posterior at or below the prior, so L scores Misled as it scores no change; Leaking counts Leak , Disclosing counts Leak and Misled .
Figure 9: Illustrative effects of six protocol choices. Each panel compares two readings of one contrast; filled markers and solid 99% intervals indicate separation from zero. Examples were selected post hoc, not sampled to estimate frequency; content proxies have no paired verdict contrast. Sources: Tables 49 (clipped vs. signed), 27 and 54 (reference, adversary), 24 (order), 21 (subject) and 53 (abstention). The second row of the reference panel and both adversary rows are read with one shared subject reference, where per-adversary intervals are reported.
System
Leak
Anon
Misled
Anonymised input
.851
.009
.140
INTACT
.874
.004
.122
Global mask
.617
.106
.277
SACC-mask
.626
.103
.272
Adaptive
.605
.141
.254
LLM rewrite †
.611
.127
.262
Appendix
Table 42: MedQA Sex entries by state ( δ=.05 ), stratum-balanced shares on the 1,473 entries the releases share; seeded orders, development adversary. † : its own document sample (1,375 entries).
Figure 10: Paired privacy–utility differences relative to SACC (star). Horizontal: content (95% for Unli , 99% for TPS); vertical: L (99% intervals), over shared entries with balanced Sex orders. Shading marks worse privacy and utility. Content is Unli on MedQA and TPS on TAB, where Unli is near its floor; axes differ across panels. I: input; M: global mask; G: global ladder; T: INTACT; A: Adaptive (same-reference entries); S: SACC-mask; R: LLM rewrite. Arrows denote values outside the scale (Adaptive on MedQA: −.237 ; input TPS minus SACC: +.155 ).
Large Language Models are increasingly applied to sensitive domains that require redaction of personally identifiable information (PII). While redacting PII is a data cleaning prerequisite, existing benchmarks conflate extraction mechanics with privacy semantics. A public phone number is not equivalent to a phone number in a medical record. Whether information constitutes a violation depends heavily on who holds it, why, and in what context, fundamentally differentiating redaction from simple entity recognition. Grounded in contextual integrity, we introduce RedactionBench, a manually annotated benchmark comprising 200 diverse documents across 11 domains, mostly seeded from real-world sources. We also introduce R-Score, a novel character-level metric that treats semantically similar redactions equally and nullifies shallow formatting choices, such as varying masking styles for phone numbers. Evaluations across Named Entity Recognition models, entity extraction Small Language Models, and frontier models equipped with agentic tools demonstrate that contextual redaction remains an unsolved problem. A human evaluation with over 80 users on RedactionBench reveals a stark dichotomy in privacy perceptions. Annotators show consensus with target labels for mandatory redactions (89.4 percent) and safe text preservations (94.1 percent), but fail to agree on contextual redactions (47.7 percent). This variance demonstrates the subjective nature of contextual privacy and motivates R-Score, which decouples contextual ambiguity from strict precision. We compare 35 models across families and report their performance in redacting PII. Finally, we release RedactionBench to establish a baseline for future privacy-preserving systems, hoping to inspire efficient model design and standardized evaluations.
Sean Brynjólfsson, Shashvat Jayakrishnan, Esha Sali +2
Authorship attribution models can re-identify users from seemingly anonymized text by exploiting stable stylistic fingerprints, even after explicit identifiers are removed, posing a growing privacy risk for text publishing and analytics. This risk extends to speech-derived text such as ASR transcripts of meetings and call-center conversations, where stylometric leakage can persist even after acoustic anonymization. Differential privacy-based anonymization often severely degrades text quality and utility. We propose a style-aware, prompt-driven anonymization approach that uses pretrained large language models to construct compact stylistic profiles from minimal samples and rewrite text to suppress identifiable style markers while preserving meaning. Across blog and review datasets, our approach reduces authorship attribution F1 by 60-70% while maintaining content quality and readability, substantially outperforming DP-based and non-DP baselines.
Current text anonymization evaluation relies on span-based metrics that fail to capture what an adversary could actually infer, and assumes a single data subject, ignoring multi-subject scenarios. To address these limitations, we present SPIA (Subject-level PII Inference Assessment), the first benchmark that shifts the unit of evaluation from text spans to individuals, comprising 675 documents across legal and online domains with novel subject-level protection metrics. Extensive experiments show that even when over 90% of PII spans are masked, subject-level inference protection drops as low as 33%, leaving the majority of personal information recoverable through contextual inference. Furthermore, target-subject-focused anonymization leaves non-target subjects substantially more exposed than the target subject. We show that subject-level inference-based evaluation is essential for ensuring safe text anonymization in real-world settings.
Myeong Seok Oh, Dong-Yun Kim, Hanseok Oh +6
Tscientific, South Korea · Soongsil University, South Korea · Yonsei University, South Korea +1