cs.AISep 18, 2025

SynBench: A Benchmark for Differentially Private Text Generation

Authors: Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar, Iqra Zahid, Yuping Wu, Yulong Wu, Hao Li, Jie Zhang, 

Organizations: Imperial College London, Imperial Global Singapore · University of Manchester, United Kingdom · CFAR and IHPC, Agency for Science, Technology and Research (A*STAR), Singapore · Nanyang Technological University, Singapore · 

Abstract

Synthetic text generation with Differential Privacy (DP) guarantees emerges as a principled approach that can enable the sharing of sensitive datasets across institutional and regulatory boundaries, while bounding the risks of re-identification and membership inference. LLM-based methods deliver promising results; however, comparisons are exacerbated by differing evaluation setups and "private" datasets, potential pre-training contamination is not considered and guarantees are not verified with DP audits. To advance this field, we introduce a unified evaluation framework with standardised utility and fidelity metrics and privacy audits, encompassing nine curated datasets that capture domain-specific complexities such as technical jargon, long-context dependencies, and specialised document structures. In a large-scale empirical study, we benchmark LLM-based state-of-the-art DP text generators of varying sizes (between 1--8B). Our results indicate that DP synthetic text generation remains an unsolved challenge, with quality deteriorating more as the private datasets deviate further from the generators' pre-training corpora. Our analyses suggest that synthetic data quality may be overestimated when LLMs have been pre-trained---without DP---on portions of the "private" data to be generated, and our membership inference (MIA) auditing setup for synthetic text reveals measurable membership signals even under tight privacy budgets. Finally, we note that under the "public pre-training, private generation" paradigm, the nominal (ε,δ)(\varepsilon,δ) guarantee does not account for exposure during pre-training, motivating contamination-aware evaluation of DP synthetic text generation. Code and data splits: https://github.com/ImperialGlobalSingapore/synbench

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ContinuousBench: Can Differentially Private Synthetic Text Improve Capabilities?

    Jun 1, 2026Peihan Liu, Lucas Rosenblatt, Weiwei Kong +7Privacy-Preserving Language ModelsSynthetic Benchmark Generation

  2. Guided Diffusion for Trustworthy Synthetic Text Generation

    Oct 19, 2026Qi Pang, Hanshen Xiao, Andre Manoel +4Synthetic Data Generation for Language ModelsDifferentially Private Synthetic Data Generation