SynBench: A Benchmark for Differentially Private Text Generation
Authors: Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar, Iqra Zahid, Yuping Wu, Yulong Wu, Hao Li, Jie Zhang, Warren Del-Pinto, Goran Nenadic, Siew Kei Lam, Anil Anthony Bharath
Organizations: Imperial College London, Imperial Global Singapore · University of Manchester, United Kingdom · CFAR and IHPC, Agency for Science, Technology and Research (A*STAR), Singapore · Nanyang Technological University, Singapore · Imperial College London, United Kingdom
Synthetic text generation with Differential Privacy (DP) guarantees emerges as a principled approach that can enable the sharing of sensitive datasets across institutional and regulatory boundaries, while bounding the risks of re-identification and membership inference. LLM-based methods deliver promising results; however, comparisons are exacerbated by differing evaluation setups and "private" datasets, potential pre-training contamination is not considered and guarantees are not verified with DP audits. To advance this field, we introduce a unified evaluation framework with standardised utility and fidelity metrics and privacy audits, encompassing nine curated datasets that capture domain-specific complexities such as technical jargon, long-context dependencies, and specialised document structures. In a large-scale empirical study, we benchmark LLM-based state-of-the-art DP text generators of varying sizes (between 1--8B). Our results indicate that DP synthetic text generation remains an unsolved challenge, with quality deteriorating more as the private datasets deviate further from the generators' pre-training corpora. Our analyses suggest that synthetic data quality may be overestimated when LLMs have been pre-trained---without DP---on portions of the "private" data to be generated, and our membership inference (MIA) auditing setup for synthetic text reveals measurable membership signals even under tight privacy budgets. Finally, we note that under the "public pre-training, private generation" paradigm, the nominal (ε,δ) guarantee does not account for exposure during pre-training, motivating contamination-aware evaluation of DP synthetic text generation. Code and data splits: https://github.com/ImperialGlobalSingapore/synbench
Figures & tables
Figure 1
Dataset
∣C∣
∣D∣
∣L∣
Access mechanism
HoC
10
10301
37/58
free on HF
N2C2’08
16
620
1985/3070
DUA & manual approval
PsyTAR
7
5102
19/34
DUA
Mimic
21
4971
2726/3951
DUA & credentialed access
DMSAFN
3
3876
30/50
free on HF
Lc37Fnd
3
5620
836/1510
free on HF
Table 1: Benchmark characteristics: Number of labels ( ∣C∣ ), dataset size ( ∣D∣ ), avg/p90 token length ∣L∣ and data access. HF: huggingface. DUA: data usage agreement.
Dataset HoC ( Random : 3.7, Majority : 9.1)
BERT-large (original=71.7 ± 2.2 )
BioClinicalBERT (original=67.6 ± 5.8 )
BiomedBERT (original=74.0 ± 0.7 )
Method
ϵ=∞
ϵ=0.5
ϵ=1
ϵ=2
ϵ=4
ϵ=∞
ϵ=0.5
ϵ=1
ϵ=2
ϵ=4
ϵ=∞
ϵ=0.5
ϵ=1
ϵ=2
ϵ=4
DP-Gen
48.2
11.2
9.7
13.0
11.6
45.3
10.0
9.5
11.9
12.2
53.0
12.7
12.6
17.0
16.8
± 3.9
± 4.7
± 3.2
± 4.0
± 0.5
± 3.6
± 2.6
± 3.7
± 2.7
± 0.5
± 1.7
± 3.6
± 4.3
± 8.9
± 2.0
AUG-PE
11.0
3.8
5.4
4.4
7.1
10.4
2.9
6.7
5.2
8.3
16.9
7.0
6.9
5.5
8.2
± 6.5
± 0.4
± 4.9
± 1.6
± 2.5
± 6.2
± 3.3
± 3.4
± 2.3
± 1.4
± 7.0
± 4.5
± 2.0
± 1.9
± 6.7
Table 2: Random/majority guess baselines, and downstream classification micro-F1 scores for three models per dataset, trained on synthetic data from Llama-3.2-1B across different privacy budgets ( ϵ ): biomedical datasets. Each value is the mean over 3 seeds (small type: ± the 95% confidence interval (t-distribution)). “–”: no result (DP-Dfs settings whose synthetic data were unusable). Bold: the best method in each column; grey: not significantly above Majority (paired bootstrap over test examples, one-sided, Benjamini-Hochberg-corrected p≥0.05 ).
Dataset DMSAFN ( Random : 30.5, Majority : 41.1)
ProsusAI-finbert (original=79.6 ± 6.4 )
finbert-tone (original=73.4 ± 2.8 )
FinancialBERT (original=92.0 ± 3.9 )
Method
ϵ=∞
ϵ=0.5
ϵ=1
ϵ=2
ϵ=4
ϵ=∞
ϵ=0.5
ϵ=1
ϵ=2
ϵ=4
ϵ=∞
ϵ=0.5
ϵ=1
ϵ=2
ϵ=4
DP-Gen
69.4
43.2
49.8
44.3
48.1
67.2
38.9
47.4
37.6
38.4
81.4
51.8
60.1
53.7
56.5
± 22.9
± 2.6
± 5.8
± 17.8
± 12.0
± 3.8
± 2.7
± 1.1
± 5.1
± 13.9
± 3.9
± 3.4
± 8.7
± 10.4
± 3.7
AUG-PE
46.4
50.2
45.4
48.5
47.7
43.9
44.8
42.1
43.1
44.2
52.6
51.1
51.6
54.4
53.4
± 1.8
± 15.5
± 3.7
± 1.3
± 1.9
± 0.4
± 4.6
± 0.3
± 0.7
± 1.6
± 1.7
± 13.0
± 8.9
± 9.3
± 8.7
Table 3: As Table 2 , for the finance and legal datasets.
Method
ϵ=∞
ϵ=4
ϵ=2
ϵ=1
ϵ=0.5
M↑ /
N↓ /
L↓
M↑ /
N↓ /
L↓
M↑ /
N↓ /
L↓
M↑ /
N↓ /
L↓
M↑ /
N↓ /
L↓
HoC
( Original : 0.99 / 2.19 / 0.004)
DP-Gen
0.65 /
3.71 /
0.06
0.18 /
5.26 /
0.40
0.16 /
5.23 /
0.417
0.14 /
5.18 /
0.44
0.12 /
5.21 /
0.47
AUG-PE
0.01 /
3.80 /
1.49
0.01 /
3.92 /
1.40
0.01 /
4.06 /
1.35
0.01 /
4.45 /
1.25
0.01 /
4.72 /
1.25
DP-Dfs
0.01 /
4.70 /
0.06
0.01 /
6.49 /
0.36
0.01 /
6.62 /
0.37
0.01 /
6.855 /
0.42
0.01 /
6.972 /
1.04
EPSVec
0.01 /
3.38 /
2.74
0.01 /
3.76 /
2.85
0.01 /
4.48 /
3.56
0.01 /
3.39 /
2.78
0.01 /
4.13 /
3.13
Table 4: Fidelity of Llama-3.2-1B synthetic data to the real test set: MAUVE ( M ), KL divergence of NER-entity 1-gram distributions ( N ) and of text-length distributions ( L ). Bold: the best method per dataset, ϵ and metric (none when all methods tie). “–”: not available (DP-Dfs settings whose synthetic data were unusable).
Figure 3: The ROC curves illustrate the performance of membership inference attacks (MIA) on synthetic data generated by AUG-PE and DP-gen. The colourful dashed lines represent the theoretical upper bounds for each corresponding privacy parameter ϵ from [ 25 ] . Zoomed sections highlight FPR ranges where the empirical ROC curve lies above the theoretical bound.
Figure 7
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Size
ε=∞
ε=4
ε=2
ε=1
ε=0.5
PsyTAR
1B
76.2 ± 0.2
35.4 ± 3.6
36.9 ± 3.2
35.2 ± 4.4
32.5 ± 2.2
3B
76.4 ± 1.0
41.1 ± 1.4
37.1 ± 2.5
31.9 ± 3.9
25.2 ± 3.9
8B
77.4 ± 0.6
43.1 ± 2.9
43.9 ± 2.2
42.0 ± 5.4
35.6 ± 1.9
N2C2’08
1B
59.1 ± 14.0
68.6 ± 1.0
69.9 ± 3.4
71.7 ± 3.6
72.7 ± 1.8
3B
65.9 ± 15.0
70.8 ± 7.8
73.0 ± 2.2
70.3 ± 3.3
70.6 ± 3.3
8B
76.4 ± 8.4
69.6 ± 1.5
70.7 ± 2.2
71.5 ± 6.2
69.5 ± 5.6
Appendix
Table 6: Micro-F1 (best classifier) of synthetic data from DP-fine-tuned Llama models of increasing size. Subscripts: 95% confidence interval over 3 seeds.
Dataset
Classifier
Original
ϵ=∞
ϵ=0.5
ϵ=1
ϵ=2
ϵ=4
HoC R 3.7 / M 9.1
BERT-large
71.7 ± 2.2
65.3 ± 1.8
27.6 ± 6.4
25.0 ± 2.9
25.2 ± 4.1
25.5 ± 6.1
BioClinicalBERT
67.6 ± 5.8
61.4 ± 3.3
25.9 ± 2.5
23.8 ± 2.8
25.9 ± 2.2
22.0 ± 2.9
BiomedBERT
74.0 ± 0.7
65.4 ± 1.3
29.4 ± 4.0
26.4 ± 2.0
29.0 ± 1.0
28.6 ± 1.9
N2C2’08 R 46.1 / M 53.2
LongFormer-large
82.6 ± 14.5
68.9 ± 22.0
55.9 ± 8.4
54.0 ± 6.5
55.1 ± 5.5
56.0 ± 12.6
Clinical-BigBird
87.4 ± 2.4
72.7 ± 6.2
52.0 ± 12.4
59.6 ± 1.9
52.0 ± 10.9
50.3 ± 10.5
Clinical-LongFormer
77.9 ± 43.5
73.6 ± 9.0
52.4 ± 1.0
54.9 ± 2.1
53.1 ± 11.9
54.4 ± 4.5
Appendix
Table 7: Downstream classification micro-F1 scores with synthetic data from OLMo-2-0425-1B (fine-tuned for 10 epochs) generated with DP-Gen, across privacy budgets ( ϵ ); “Original” trains on the real data. R / M: random / majority guess baselines. Each value is the mean over 3 seeds ± the 95% confidence interval (t-distribution).
Figure 5: Utility of OLMo-2-1B DP synthetic data ( ε=4 ) for HoC against the share x of the private training set ( N=4,000 ) that OLMo-2 saw in mid-training (dolmino-mix-1124). Each point is one DP generator (303 from three independent replicates), scored by a classifier trained on its synthetic data and tested on the real HoC test set, as the mean or maximum over 3 classifier seeds. Micro-F1 is rescaled per classifier to the percentage of the gain over random guessing (0.037) that the same classifier achieves when trained on the real 4,000 records (0.719 BiomedBERT, 0.712 BioLinkBERT). Red: least-squares fit; p : one-sided permutation test of a positive slope.
Classifier (seeds)
x=0→100
Δ
p
p (collapsed =0 )
BiomedBERT (mean)
41.9 → 43.2
+1.3
0.002
0.003
BiomedBERT (max)
43.7 → 45.2
+1.5
0.0009
0.0009
BioLinkBERT (mean)
39.4 → 42.8
+3.4
< 0.0001
0.009
BioLinkBERT (max)
42.9 → 45.4
+2.5
0.0002
0.0003
all four (overall)
< 0.0001
0.0001
Appendix
Table 8: Rescaled downstream micro-F1 (%) of OLMo-2-1B DP synthetic data ( ε=4 , 303 generators) for HoC as the share x of private records seen in OLMo-2 mid-training grows from 0% to 100% ( N=4,000 ): least-squares fit at the two ends, its change, and one-sided permutation p of a positive slope (20,000 shuffles of x within each of the three replicates). Collapsed classifier runs (micro-F1 0) are left out, or counted as 0 in the last column. Overall: one test of all four views at once (weakest standardised slope).
Dataset
Regime
RMIA
LiRA
Logits
DMSAFN
SYN (No DP)
0.60
0.49
0.60
DMSAFN
SYN ( ϵ=2 )
0.54
0.45
0.48
DMSAFN
PRI (No DP)
0.45
0.50
0.54
DMSAFN
PRI ( ϵ=2 )
0.49
0.49
0.54
PsyTAR
SYN (No DP)
0.65
0.61
0.62
PsyTAR
SYN ( ϵ=2 )
0.57
0.61
0.61
Appendix
Table 9: MIA AUC across attacks and training regimes. SYN denotes a downstream classifier trained on synthetic data; PRI denotes training on private data. “No DP” vs. ϵ=2 refers to whether DP is applied in the relevant training/generation step described in Section 4 . Higher AUC indicates stronger membership distinguishability ( 0.5 is chance).
Figure 6: Empirical ε implied by the bigram reference attack along its ROC (mean over 50 repeats; bands: 2.5–97.5% quantiles; dashed: guaranteed ε ). Hatched: α<0.1 or α>0.9 , where the estimate rests on ≤ 5 records per repeat. Circles: the 2.5% quantile exceeds the guaranteed ε .
Figure 7: Metric trends across epsilon levels for AUG-PE and DP-Gen Methods. Each subplot shows how a specific metric changes with increasing epsilon levels. Solid blue lines represent AUG-PE and dashed orange lines represent DP-Gen, with shaded areas indicating variance across datasets. Metrics are reported across 9 datasets.