Synthetic Data Evaluation

Ask about this topic

Overview

Synthetic data evaluation asks whether generated examples are realistic and suitable for their intended use, such as model training, scientific inference, or benchmarking. Researchers combine distributional and dependence metrics with train-on-synthetic, test-on-real experiments, human or expert review, domain-specific checks, and privacy attacks. Controlled benchmarks with known structure can test whether methods preserve causal, temporal, subgroup, or task-relevant properties that ordinary summary statistics may miss. A central open challenge is that no single fidelity score reliably predicts utility or privacy: results depend on the task, data domain, and evaluation protocol.

Latest papers

CardsList

236 papers26 papers in the last four weeks, down 19% on the four weeks before. 0.2% of all new papers.

  1. Multimodal Dataset Normalization and Perceptual Validation for Music-Taste Correspondences

    Apr 12, 2026Matteo Spanio, Valentina Frezzato, Antonio RodàSynthetic Data Evaluation

    Department of Information Engineering, University of Padova, Via Gradenigo 6/B, 35131 Padova, Italy

  2. Synthesizing real-world distributions from high-dimensional Gaussian Noise with Fully Connected Neural Network

    Apr 10, 2026Joanna KomorniczakSynthetic Tabular Data GenerationGenerative Modeling

    Department of Systems and Computer Networks, Wrocław University of Science and Technology

  3. Validated Synthetic Patient Generation for Small Longitudinal Cohorts: Coagulation Dynamics Across Pregnancy

    Apr 8, 2026Jeffrey D. Varner, Maria Cristina Bravo, Carole McBride +2Synthetic Data GenerationSynthetic Tabular Data Generation

    Robert F. Smith School of Chemical and Biomolecular Engineering, Cornell University, Ithaca, NY 14850 · Department of Biochemistry, The Robert Larner, M.D. College of Medicine, University of Vermont Medical Center, Burlington, VT 05401 · Department of Obstetrics, Gynecology, and Reproductive Sciences, The Robert Larner, M.D. College of Medicine, University of Vermont Medical Center, Burlington, VT 05401

  4. Stable and Privacy-Preserving Synthetic Educational Data with Empirical Marginals: A Copula-Based Approach

    Apr 5, 2026Gabriel Diaz Ramos, Lorenzo Luzi, Debshila Basu Mallick +1Differentially Private Synthetic Data GenerationSynthetic Data Generation

    Rice University · OpenStax · Rice University, OpenStax

  5. RDFace: A Benchmark Dataset for Rare Disease Facial Image Analysis under Extreme Data Scarcity and Phenotype-Aware Synthetic Generation

    Apr 3, 2026Ganlin Feng, Yuxi Long, Hafsa Ali +5Rare Disease DiagnosisSynthetic Data Evaluation

    Western University · Concordia University · University of Toronto +1

  6. Statistical realism is not evidence that LLMs can estimate treatment effects in social science experiments

    Apr 2, 2026Zonghan Li, Feng JiLarge Language Model-Based Social SimulationSynthetic Data Evaluation

    Department of Applied Psychology and Human Development, University of Toronto, ON, Canada

  7. Know Your Streams: On the Conceptualization, Characterization, and Generation of Intentional Event Streams

    Apr 1, 2026Andrea Maldonado, Christian Imenkamp, Hendrik Reiter +4Process MiningSynthetic Data Evaluation

    School of Engineering and Design, Technical University of Munich, Germany · Database Systems and Data Mining, Ludwig Maximilian University of Munich, Germany · Munich Center for Machine Learning, Munich, Germany +2

  8. Optimsyn: Influence-Guided Rubrics Optimization for Synthetic Data Generation

    Apr 1, 2026Zhiting Fan, Ruizhe Chen, Tianxiang Hu +7Synthetic Data GenerationSynthetic Data Evaluation

    Zhejiang University · Inclusion AI, Ant Group · Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence

  9. Reasoning-Driven Synthetic Data Generation and Evaluation

    Mar 31, 2026Tim R. Davidson, Benoit Seguin, Enrico Bacis +2Synthetic Data GenerationSynthetic Data Evaluation

    EPFL · Google · Google Deepmind

  10. Leveraging Synthetic Data for Enhancing Egocentric Hand-Object Interaction Detection

    Mar 31, 2026Rosario Leonardi, Antonino Furnari, Francesco Ragusa +1Hand-Object Interaction DetectionHuman-Object Interaction

    Department of Mathematics and Computer Science, University of Catania, Catania, Italy. · Next Vision s.r.l., Catania, Italy.

  11. Synthetic Designed Experiments for Diagnosing Vision Model Failure

    Mar 30, 2026Krisanu SarkarSynthetic Data EvaluationAlgorithmic Auditing

    Indian Institute of Technology Bombay, Mumbai, India

  12. Amalgam: Hybrid LLM-PGM Synthesis Algorithm for Accuracy and Realism

    Mar 28, 2026Antheas Kapenekakis, Bent Thomsen, Katja Hose +1Synthetic Tabular Data GenerationGenerative Modeling

    Aalborg University, Aalborg, Denmark · TU Wien, Vienna, Austria

  13. Comparing Natural and Synthetic Structured Data: A Study of the Passive Verb Alternation in French and Italian

    Mar 26, 2026Giuseppe Samo, Paola MerloSynthetic Data EvaluationSynthetic Data Pretraining

    IDIAP Research Institute · University of Geneva

  14. Evaluating Synthetic Images as Effective Substitutes for Experimental Data in Surface Roughness Classification

    Mar 26, 2026Binwei Chen, Huachao Leng, Chi Yeung Mang +5Synthetic Data EvaluationSynthetic Data Augmentation

    Division of Science, Engineering and Health Studies, The School of Professional Education and Executive Development, The Hong Kong Polytechnic University, Hong Kong, China · State Key Laboratory of Ultra-precision Machining Technology, Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, China · Department of Science and Environmental Studies, The Education University of Hong Kong, Hong Kong, China

  15. Synthetic Cardiac MRI Image Generation using Deep Generative Models

    Mar 25, 2026Ishan Kumarasinghe, Dasuni Kawya, Madhura Edirisooriya +3Synthetic Data GenerationMedical Image Synthesis

    Department of Computer Engineering, University of Peradeniya, Peradeniya, Sri Lanka · Department of Computer Engineering, University of Maryland, College Park, USA · Department of Holistic Systems, SimulaMet, Oslo, Norway

  16. Generating High Quality Synthetic Data for Dutch Medical Conversations

    Mar 25, 2026Cecilia Kuan, Aditya Kamlesh Parikh, Henk van den HeuvelSynthetic Data Generation for Language ModelsText Generation

    Center for Language Studies, Radboud University, Nijmegen, Netherlands

  17. A Sobering Look at Tabular Data Generation via Probabilistic Circuits

    Mar 24, 2026Davide Scassola, Dylan Ponsford, Adrián Javaloy +4Synthetic Tabular Data GenerationProbabilistic Circuits

    AILAB, University of Trieste, Trieste, Italy · Aindo SpA, AREA Science Park, Trieste, Italy · School of Informatics, University of Edinburgh, Edinburgh, UK

  18. Generating Synthetic Wildlife Health Data from Camera Trap Imagery: A Pipeline for Alopecia and Body Condition Training Data

    Mar 23, 2026David BrundageSynthetic Data GenerationSynthetic Data Evaluation

    University of Wisconsin - Madison, School of Veterinary Medicine

  19. MIDST Challenge at SaTML 2025: Membership Inference over Diffusion-models-based Synthetic Tabular data

    Mar 19, 2026Masoumeh Shafieinejad, Xi He, Mahshid Alinoori +6Synthetic Data EvaluationMembership Inference Attacks

    Vector Institute · University of Waterloo · McGill University

  20. Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generation

    Mar 18, 2026Iakovos-Christos Zarkadis, Christos DouligerisNetwork Intrusion DetectionSynthetic Data Generation

    University of Piraeus, Athens, Greece · Dept. of Informatics, University of Piraeus, Piraeus, Greece

  21. When Marginals Match but Structure Fails: Covariance Fidelity in Generative Models

    Mar 17, 2026Nazia RiasatSynthetic Data EvaluationGenerative Modeling

    North Dakota State University

  22. Who Benchmarks the Benchmarks? A Case Study of LLM Evaluation in Icelandic

    Mar 17, 2026Finnur Ágúst Ingimundarson, Steinunn Rut Friðriksdóttir, Bjarki Ármannsson +2LLM EvaluationBenchmark Validity

    University of Zürich · University of Iceland · The Árni Magnússon Institute for Icelandic Studies

  23. REAEDP: Entropy-Calibrated Differentially Private Data Release with Formal Guarantees and Attack-Based Evaluation

    Mar 14, 2026Bo Ma, Jinsong Wu, Wei Qi YanDifferentially Private Synthetic Data GenerationDifferential Privacy

    School of Mathematics and Computer Engineering, Auckland University of Technology, Auckland 1024 New Zealand · Resideo Technologies, Inc., Austin, TX 78759, USA · Department of Artificial Intelligence, Guilin University of Electronic Technology, Gui Lin, China +1

  24. Design and evaluation of an agentic workflow for crisis-related synthetic tweet datasets

    Mar 13, 2026Roben Delos Reyes, Timothy Douglas, Asanobu KitamotoSynthetic Data GenerationAgentic Workflows

    School of Computing and Information Systems, The University of Melbourne, Parkville, Victoria, Australia. · National Institute of Informatics, Tokyo, Japan. · Department of Computer Science, University College London, London, United Kingdom.

  25. Synthetic Melanoma Image Generation and Evaluation Using Generative Adversarial Networks

    Mar 13, 2026Pei-Yu Lin, Yidan Shen, Neville Mathew +6Medical Image SynthesisSynthetic Data Evaluation

    Department of Engineering Technology, University of Houston, Sugar Land, TX 77479, USA · Department of Electrical and Computer Engineering, University of Houston, Houston, TX 77204, USA · Department of Information Science Technology, University of Houston, Sugar Land, TX 77479, USA +5

  26. QAQ: Bidirectional Semantic Coherence for Selecting High-Quality Synthetic Code Instructions

    Mar 12, 2026Jiayin Lei, Ming Ma, Yunxi Duan +2Instruction-Tuning Data SelectionCode Generation

    Beijing University of Technology, Beijing, 100124, China · Institute of Neuroscience, State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China · University of Chinese Academy of Sciences, Beijing, 100049, China +1

  27. Synthetic Data Generation for Brain-Computer Interfaces: Overview, Benchmarking, and Future Directions

    Mar 11, 2026Ziwei Wang, Zhentao He, Xingyi He +6Synthetic Data GenerationBrain-Computer Interfaces

    School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China · Zhongguancun Academy, Beijing, China

  28. A Standardized Framework For Evaluating Gene Expression Generative Models

    Mar 11, 2026Andrea Rubbi, Andrea Giuseppe Di Francesco, Mohammad Lotfollahi +1Synthetic Data EvaluationSingle-Cell Transcriptomics

    Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom · Wellcome Sanger Institute, Cambridge, United Kingdom · Sapienza University of Rome, Rome, Italy +3

  29. Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators

    Mar 11, 2026Rajdeep Pathak, Amit Basak, Sayantee JanaMembership Inference AttacksSynthetic Data Evaluation

    SCAI, Sorbonne Université, Paris, France · SAFIR, Sorbonne University Abu Dhabi, UAE · Indian Institute of Technology Hyderabad, India

  30. Grounding Synthetic Data Generation With Vision and Language Models

    Mar 10, 2026Ümit Mert Çağlar, Alptekin TemizelSynthetic Data AugmentationSynthetic Data Evaluation

    Graduate School of Informatics METU, Turkey