Synthetic Data Evaluation

Ask about this topic

Overview

Synthetic data evaluation asks whether generated examples are realistic and suitable for their intended use, such as model training, scientific inference, or benchmarking. Researchers combine distributional and dependence metrics with train-on-synthetic, test-on-real experiments, human or expert review, domain-specific checks, and privacy attacks. Controlled benchmarks with known structure can test whether methods preserve causal, temporal, subgroup, or task-relevant properties that ordinary summary statistics may miss. A central open challenge is that no single fidelity score reliably predicts utility or privacy: results depend on the task, data domain, and evaluation protocol.

Latest papers

CardsList

236 papers26 papers in the last four weeks, down 19% on the four weeks before. 0.2% of all new papers.

  1. Sequential versus fully conditional specification for tree-based synthesis of health data

    Oct 26, 2026R{é}my Chapelle, Mohammed SedkiSynthetic Clinical Data GenerationTabular Data Synthesis

    CB, CESP, EVDG · Université Paris-Saclay, UVSQ, Inserm, CESP, 94807, Villejuif, France · Université Paris-Saclay, Université Paris Cité, ENS Paris-Saclay, CNRS, SSA, Inserm, Centre Borelli, 91190, Gif-sur-Yvette, France +2

  2. Synthetic Child-Adult Interactions Are Not Developmentally Plausible

    Oct 26, 2026Bastian Bunzeck, Francesca PadovaniLanguage ModelingSynthetic Data Evaluation

    Computational Linguistics, Department of Linguistics, Bielefeld University · Center for Language and Cognition (CLCG), University of Groningen

  3. TwinStore: A Synthetic Consumer Platform Using Vision--Language Models

    Oct 22, 2026Minhyeok LeeSynthetic Data GenerationDigital Twins

  4. Silicon Samples Match the Answers but Not the Respondents

    Oct 22, 2026Yusuke Kumagae, Masahiro KanekoSynthetic Population GenerationSynthetic Data Evaluation

    Hakuhodo DY Holdings Inc. · MBZUAI

  5. OSynth: A Longitudinal, Context-Sensitive Evaluation of Meeting Understanding

    Oct 21, 2026Ian Porada, Suraj Tripathi, Chaitanya Animesh +8Synthetic Data GenerationSynthetic Data Evaluation

    Otter.ai Mountain View, California, USA

  6. Not All Similarity Is Leakage: Distribution-Aware Identity Leakage Assessment in Synthetic Face Datasets

    Oct 19, 2026Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo +3Data LeakageSynthetic Data Evaluation

    University of Luxembourg, Luxembourg · Fraunhofer IGD, Darmstadt, Germany

  7. FinSafeGuard: A Quality-Driven Synthetic Dataset for Financial AI Safety

    Oct 19, 2026Joseph Thomas Thacil, Reetu Raj Harsh, Krati Saxena +2Synthetic Data GenerationSynthetic Data Evaluation

    Domyn, India · Domyn, USA

  8. Exploring The Empirical Privacy Landscape of Synthetic Data

    Oct 15, 2026Zihao Zhao, Avi Schwarzschild, Casey Meehan +3Privacy Leakage in Language ModelsPrivacy-Preserving Language Models

    Johns Hopkins University · OpenAI

  9. GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty

    Oct 15, 2026Gaocheng Yang, Ruijie Tao, Kai Ye +3Synthetic Data Generation for LLM AgentsSynthetic Data Generation for Language Models

    Johns Hopkins University · Nanjing University · Amazon

  10. LURE: a Phase-Labelled Synthetic Corpus of Online-Grooming Exchanges

    Oct 11, 2026Emanuele Florindi, Valentina Franzoni, Ludovico Guercio +2Synthetic Data GenerationSynthetic Data Evaluation

    the LESDIT PhD School, Learning Science and Digital Technologies, University of Modena and Reggio Emilia, Reggio Emilia, Italy · the EMORE Lab, Department of Mathematics and Computer Science, University of Perugia, 06123 Perugia, Italy · the Department of Computer Science, Hong Kong Baptist University, Kowloon Tong, Hong Kong +1

  11. CoverSynth: Coverage-Guided Generation of Retrieval Evaluation Sets

    Oct 11, 2026Anahita Sharma, Apaar Shanker, Rishav Chakravarti +6Synthetic Benchmark GenerationIR Evaluation

    Scale AI

  12. PATTERN Can Be Solved from Memory: Generator-Aware Baselines for Synthetic Node Classification

    Oct 10, 2026Amin Najafgholizadeh, Mohsen AfsharchiBenchmark ContaminationSynthetic Data Evaluation

    University of Zanjan

  13. Probable but Not Valid: A Psychometric Audit of Decision Models as Synthetic Survey Respondents

    Oct 10, 2026Mantas Lukauskas, Viktorija ŠarkauskaitėSynthetic Data GenerationSynthetic Data Evaluation

    Hostinger; Kaunas University of Technology; AI Insight Lab · AI Insight Lab; Independent Researcher

  14. Can Synthetic UAV Benchmarks Be Trusted? Real-Synthetic Validation for UAV Model Selection

    Oct 10, 2026Yuyi Zhi, Min Du, Bill Shi +4Synthetic Data EvaluationUAV Remote Sensing

    Yuyi Zhi is with Zhejiang University. · Min Du, Bill Shi, Guodong Yuan, and Xin Guan are with Avant Robotics. · Junheng Liang is with University of Science and Technology Beijing. +1

  15. MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis

    Oct 8, 2026Ximing Lu, Aviral Kumar, Jan Kautz +2Reinforcement Learning with Verifiable RewardsSynthetic Data Generation for Language Models

    University of Washington · Carnegie Mellon University · NVIDIA +1

  16. Intervention anchors and scientific verification in synthetic vascular predictive representations

    Oct 8, 2026Lingsen You, Yujun Guo, Xinyu Zhong +4Representation IdentifiabilityPredictive Representation Learning

    Department of Cardiology, Zhongshan Hospital, Fudan University, Shanghai Institute of Cardiovascular Diseases, State Key Laboratory of Cardiovascular Diseases, NHC Key Laboratory of Ischemic Heart Diseases, National Clinical Research Center for Interventional Medicine, Key Laboratory of Viral Heart Diseases, Chinese Academy of Medical Sciences, Shanghai 200032, China. · Institutes of Biomedical Sciences, Fudan University, Shanghai 200030, China. · College of Biomedical Engineering, Fudan University, Shanghai 200438, China. +1

  17. Conditional Transfer from Controlled Pretraining Mixtures to Code

    Oct 8, 2026Ohad RubinSynthetic Data PretrainingData Mixture Optimization

  18. Beyond Distributional Fidelity: Causal-Penalized Diffusion for Synthetic Tabular Data

    Oct 8, 2026Lan Tao, Yongxian He, Shirong Xu +2Synthetic Tabular Data GenerationTabular Diffusion Models

    Department of Statistics & Data Science, University of California, Los Angeles · Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego · Wang Yanan Institute for Studies in Economics, Xiamen University

  19. Evaluating Sequence Assembly Strategies for Differentially Private Synthetic Time-Series Forecasting

    Oct 7, 2026Guoxiong Long, Huizhen Huang, Qikun Cai +2Synthetic Data EvaluationSynthetic Data Generation

    School of Computer and Big Data, Minjiang University, No. 200 Xiyuangong Road, Fuzhou, 350108, Fujian, China

  20. FaceKit: a Toolkit for Interpretable Facial Phenotyping, Synthetic Image Generation and Privacy Analysis in Rare Diseases

    Oct 7, 2026Hongzhuo Chen, Zhanliang Wang, Florent Pollet +9Rare Disease DiagnosisSynthetic Data Evaluation

    Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children’s Hospital of Philadelphia, Philadelphia, PA, USA · Applied Mathematics and Computational Science Graduate Program, University of Pennsylvania, Philadelphia, PA, USA · Department of Biomedical Informatics, Columbia University, New York, NY, USA +6

  21. Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs

    Oct 6, 2026Ankush CheckervartyLanguage Model ProbingSynthetic Data Evaluation

    Independent Researcher

  22. Below what training size do deep tabular generators stop beating trivial baselines? A preregistered benchmark on a size ladder of clinical and standard datasets

    Oct 2, 2026Shivam ShrivastavaSynthetic Tabular Data GenerationSynthetic Data Evaluation

    VIT Bhopal University, India

  23. Invent a Dataset: Measuring Dataset Generation Abilities with Zero Seed Data

    Oct 1, 2026Shivalika Singh, Andrija Djurisic, Gbemileke Onilude +2Synthetic Data GenerationSynthetic Data Generation for Language Models

    Adaption

  24. Generation Provenance Before Behavior Attribution: Auditing Synthetic Speech Research Objects

    Oct 1, 2026Sidi Chang, Peiying ZhuData ProvenanceTraining Data Attribution

    Blossom AI Labs Blossom AI Tokyo, Japan

  25. Source Identification Is Not Fitness Testing: Measuring the Limits of Synthetic-Data Attribution

    Sep 30, 2026Joss ArmstrongModel CollapseTraining Data Attribution

    Ericsson, Athlone, Ireland

  26. Synthetic Data Characterization via Training Dynamics

    Sep 30, 2026Irene Lago, Ana Ezquerro, David VilaresSynthetic Data Generation for Language ModelsSynthetic Data Evaluation

    Universidade da Coruña, CITIC, Spain · Graz University of Technology, IML, Austria

  27. SimTrace: Grounded Multimodal User Trajectories Generation for Online User Modeling

    Sep 29, 2026Yunan Lu, Shuang Xie, Meghna Allamudi +4Synthetic Data GenerationUser Modeling

    Shopify · Columbia University

  28. AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?

    Sep 28, 2026Haotian Luo, Haoyu Wang, Zeyu Qin +5AI Agent BenchmarksSynthetic Benchmark Generation

    HKUST · SLAI · NTU

  29. Marginal Calibration Masks Generator Differences in Held-Out Evaluation of Multivariate Time Series, with an Application to ERA5 Weather Sequences

    Sep 27, 2026Mustafa Özaytaç, Özge Karadağ AtaşTime Series GenerationModel Calibration

    Department of Statistics, Graduate School of Science and Engineering, Hacettepe University, Ankara, 06800, Turkey · Department of Statistics, Faculty of Science, Hacettepe University, Ankara, 06800, Turkey

  30. Shared Autoregressive Context Can Distort Relationships in Synthetic Data

    Sep 26, 2026Thomas S. RobinsonSynthetic Data Generation for Language ModelsSynthetic Data Evaluation

    London School of Economics and Political Science