Synthetic Data

Follow this topic Ask about this topic

Momentum

48 papers in the last four weeks, up 92% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 406

Apr 10, 2026cs.CV

Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory

With the advancement of interactive video generation, diffusion models have increasingly demonstrated their potential as world models. However, existing approaches still struggle to simultaneously achieve memory-enabled long-term temporal consistency and high-resolution real-time generation, limiting their applicability in real-world scenarios. To address this, we present Matrix-Game 3.0, a memory-augmented interactive world model designed for 720p real-time longform video generation. Building upon Matrix-Game 2.0, we introduce systematic improvements across data, model, and inference. First, we develop an upgraded industrial-scale infinite data engine that integrates Unreal Engine-based synthetic data, large-scale automated collection from AAA games, and real-world video augmentation to produce high-quality Video-Pose-Action-Prompt quadruplet data at scale. Second, we propose a training framework for long-horizon consistency: by modeling prediction residuals and re-injecting imperfect generated frames during training, the base model learns self-correction; meanwhile, camera-aware memory retrieval and injection enable the base model to achieve long horizon spatiotemporal consistency. Third, we design a multi-segment autoregressive distillation strategy based on Distribution Matching Distillation (DMD), combined with model quantization and VAE decoder pruning, to achieve efficient real-time inference. Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model, while maintaining stable memory consistency over minute-long sequences. Scaling up to a 2x14B model further improves generation quality, dynamics, and generalization. Our approach provides a practical pathway toward industrial-scale deployable world models.

Zile Wang, Zexiang Liu, Jiaxing Li +20

Skywork AI

Video World Models World Models Real-Time Systems Diffusion Models Synthetic Data

Apr 8, 2026cs.CR

Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation

Large language models (LLMs) have emerged as a powerful tool for synthetic data generation. A particularly important use case is producing synthetic replicas of private text, which requires carefully balancing privacy and utility. We propose Realistic and Privacy-Preserving Synthetic Data Generation (RPSG), which uses private seeds and integrates privacy-preserving strategies, including a formal differential privacy (DP) mechanism in the candidate selection, to generate realistic synthetic data. Comprehensive experiments against state-of-the-art private synthetic data generation methods demonstrate that RPSG achieves high fidelity to private data while providing strong privacy protection.

Qian Ma, Sarah Rajtmajer

Information Sciences and Technology, The Pennsylvania State University

Synthetic Data Δ)$-Differential Privacy

Apr 6, 2026cs.CV

SimpleProc: Fully Procedural Synthetic Data from Simple Rules for Multi-View Stereo

Generating procedural synthetic data for multi-view stereo (MVS) usually requires writing complex rules to match the realism of curated datasets. We demonstrate that we can generate effective training data using SimpleProc: a new, fully procedural generator driven by a very small set of rules based on Non-Uniform Rational Basis Splines (NURBS), as well as simple displacement and texture patterns. At a modest scale of 8,000 images, our approach achieves superior results compared to manually curated images at the same scale sourced from games and real-world objects. When scaled to 352,000 images, our approach achieves similar, and in some benchmarks, even better results than the current state-of-the-art trained on over 692,000 manually curated images. The source code and the data are available at https://github.com/princeton-vl/SimpleProc.

Zeyu Ma, Alexander Raistrick, Jia Deng

Princeton University

Large-Scale 3D Editing Dataset Synthetic Data Multi-View Objects Simple Splines

Mar 30, 2026cs.CV

PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models

Acquiring labeled datasets for 3D human mesh estimation is challenging due to depth ambiguities and the inherent difficulty of annotating 3D geometry from monocular images. Existing datasets are either real, with manually annotated 3D geometry and limited scale, or synthetic, rendered from 3D engines that provide precise labels but suffer from limited photorealism, low diversity, and high production costs. In this work, we explore a third path: generated data. We introduce PoseDreamer, a novel pipeline that leverages diffusion models to generate large-scale synthetic datasets with 3D mesh annotations. Our approach combines controllable image generation with Direct Preference Optimization for control alignment, curriculum-based hard sample mining, and multi-stage quality filtering. Together, these components naturally maintain correspondence between 3D labels and generated images, while prioritizing challenging samples to maximize dataset utility. Using PoseDreamer, we generate more than 500,000 high-quality synthetic samples, achieving a 76% improvement in image-quality metrics compared to rendering-based datasets. Models trained on PoseDreamer achieve performance comparable to or superior to those trained on real-world and traditional synthetic datasets. In addition, combining PoseDreamer with synthetic datasets results in better performance than combining real-world and synthetic datasets, demonstrating the complementary nature of our dataset. We will release the full dataset and generation code.

Lorenza Prospero, Orest Kupyn, Ostap Viniavskyi +2

The Podium Institute for Sports Medicine and Technology University of Oxford Oxford, UK · Visual Geometry Group University of Oxford Oxford, UK · Ukrainian Catholic University Lviv, Ukraine

Synthetic Data Diffusion Models

Mar 24, 2026cs.RO

Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Models

Learning a generalist control policy for robotic manipulation typically relies on large-scale datasets. Given the high cost of real-world data collection, a practical alternative is to generate synthetic data through simulation. However, the resulting synthetic data often exhibits a significant gap from real-world distributions. While many prior studies have proposed algorithms to bridge the Sim-to-Real discrepancy, there remains a lack of principled research that grounds these methods in real-world manipulation tasks, particularly their performance on generalist policies such as Vision-Language-Action (VLA) models. In this study, we empirically examine the primary determinants of Sim-to-Real generalization across four dimensions: multi-level domain randomization, photorealistic rendering, physics-realistic modeling, and reinforcement learning updates. To support this study, we design a comprehensive evaluation protocol to quantify the real-world performance of manipulation tasks. The protocol accounts for key variations in background, lighting, distractors, object types, and spatial features. Through experiments involving over 10k real-world trials, we derive critical insights into Sim-to-Real transfer. To inform and advance future studies, we release both the robotic platforms and the evaluation protocol for public access to facilitate independent verification, thereby establishing a realistic and standardized benchmark for robotic manipulation policies.

Ruixing Jin, Zicheng Zhu, Ruixiang Ouyang +4

School of Data Science, The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute

Robotic Manipulation Policies Robotic Manipulation Sim-To-Real Gap Real-World Manipulation Tasks Synthetic Data

Mar 18, 2026cs.CV

FMMC: Harnessing the Power of Foundation Models for Accurate Material Classification

Material classification has emerged as a critical task in computer vision and graphics, supporting the assignment of accurate material properties to a wide range of digital and real-world applications. While traditionally framed as an image classification task, this domain faces significant challenges due to the scarcity of annotated data, limiting the accuracy and generalizability of trained models. Recent advances in vision-language foundation models (VLMs) offer promising avenues to address these issues, yet existing solutions leveraging these models still exhibit unsatisfying results in material recognition tasks. In this work, we propose a novel framework that effectively harnesses foundation models to overcome data limitations and enhance classification accuracy. Our method integrates two key innovations: (a) a robust image generation and auto-labeling pipeline that creates a diverse and high-quality training dataset with material-centric images, and automatically assigns labels by fusing object semantics and material attributes in text prompts; (b) a prior incorporation strategy to distill information from VLMs, combined with a joint fine-tuning method that optimizes a pre-trained vision foundation model alongside VLM-derived priors, preserving broad generalizability while adapting to material-specific features. Extensive experiments demonstrate significant improvements on multiple datasets. We show that our synthetic dataset effectively captures the characteristics of real world materials, and the integration of priors from vision-language models significantly enhances the final performance. The source code and dataset will be released.

Qingran Lin, Fengwei Yang, Chaolun Zhu

Georgia Institute of Technology · Duke University · Waseda University

Recent Vision Foundation Models Materials Vision-Language Foundation Models Foundation Model Synthetic Data

Mar 15, 2026cs.LG

Deep probabilistic model synthesis enables unified modeling of whole-brain neural activity across individual subjects

Many disciplines need quantitative models that synthesize experimental data across multiple instances of the same general system. For example, neuroscientists must combine data from the brains of many individual animals to understand the species' brain in general. However, typical machine learning models treat one system instance at a time. Here we introduce a machine learning framework, deep probabilistic model synthesis (DPMS), that leverages system properties auxiliary to the model to combine data across system instances. DPMS specifically uses variational inference to learn a conditional prior distribution and instance-specific posterior distributions over model parameters that respectively tie together the system instances and capture their unique structure. DPMS can synthesize a wide variety of model classes, such as those for regression, classification, and dimensionality reduction, and we demonstrate its ability to improve upon single-instance models on synthetic data and whole-brain neural activity data from larval zebrafish.

William E. Bishop, Luuk W. Hesselink, Bernhard Englitz +2

Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA · Computational Neuroscience Lab, Donders Center for Neuroscience, Radboud University, Nijmegen, The Netherlands · Departments of Neurobiology, Engineering Sciences and Applied Mathematics, and Physics and Astronomy, Northwestern University, Evanston, IL +1

Morphogenesis Synthetic Data Posterior Controlled Comparison

Mar 12, 2026cs.CL

QAQ: Bidirectional Semantic Coherence for Selecting High-Quality Synthetic Code Instructions

Synthetic data has become essential for training code generation models, yet it introduces significant noise and hallucinations that are difficult to detect with current metrics. Existing data selection methods like Instruction-Following Difficulty (IFD) typically assess how hard a model generates an answer given a query (

A|Q

). However, this metric is ambiguous on noisy synthetic data, where low probability can distinguish between intrinsic task complexity and model-generated hallucinations. Here, we propose QAQ, a novel data selection framework that evaluates data quality from the reverse direction: how well can the answer predict the query (

Q|A

)? We define Reverse Mutual Information (RMI) to quantify the information gain about the query conditioned on the answer. Our analyses reveal that both extremes of RMI signal quality issues: low RMI indicates semantic misalignment, while excessively high RMI may contain defect patterns that LLMs easily recognize. Furthermore, we introduce a selection strategy based on the disagreement between strong and weak models to identify samples that are valid yet challenging. Experiments across three datasets spanning code generation (WarriorCoder, Magpie-Qwen2.5-Coder-Pro-300K) and math reasoning (OpenR1-Math-220k) demonstrate that selecting just 25% of data using stratified RMI matches full-data performance while being consistently competitive with or better than existing data selection methods. Our approach highlights the importance of bidirectional semantic coherence in synthetic data curation, offering a scalable pathway to reduce computational costs without sacrificing model capability. Code is available at https://github.com/XXSg559/QAQ.

Jiayin Lei, Ming Ma, Yunxi Duan +2

Beijing University of Technology, Beijing, 100124, China · Institute of Neuroscience, State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China · University of Chinese Academy of Sciences, Beijing, 100049, China +1

Synthetic Data Data Quality Llm-Driven Code Synthesis Code Generation Instruction

Mar 11, 2026cs.CV

HyPER-GAN: Hybrid Patch-Based Image-to-Image Translation for Real-Time Photorealism Enhancement in Game Engines

Generative models are increasingly used in video game engines to enhance the photorealism of rendered images for visual synthetic data generation and simulation applications. However, they often introduce artifacts that alter the content of the original rendered scenes and require high computational resources, which limit their utilization for the photorealism enhancement of training and evaluation data, as well as their integration in the rendering pipelines of game engines. In this paper, we propose Hybrid Patch Enhanced Realism Generative Adversarial Network (HyPER-GAN), a hybrid image-to-image translation framework that is based on a lightweight U-Net-style generator capable of performing real-time inference. The framework is trained using paired rendered and photorealism-enhanced images, complemented by a novel hybrid training strategy that incorporates matched patches from unpaired real-world images to improve content preservation and further enhance the visual realism that can be achieved by the lightweight generator. Experimental results demonstrate that HyPER-GAN achieves a 6x increase in frames per second at 1080p in comparison with state-of-the-art lightweight paired image-to-image translation methods, while also increasing, in both within- and cross-engine evaluations, the photorealism of the rendered images without significantly compromising semantic consistency. Moreover, it is illustrated that HyPER-GAN maintains temporal consistency and that the proposed hybrid training strategy improves content preservation and visual realism in within-engine and increases the robustness in cross-engine evaluations compared to training the framework solely with paired rendered and photorealism-enhanced images. Code and pretrained models are publicly available at: https://github.com/stefanos50/HyPER-GAN

Stefanos Pasios, Nikos Nikolaidis

School of Informatics, Aristotle University of Thessaloniki (AUTH), Thessaloniki, 54124, Greece

Unpaired Image-To-Image Translation Generative Adversarial Network Synthetic Data

Mar 10, 2026cs.LG

Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure

Synthetic tabular data generation addresses data scarcity and privacy constraints in a variety of domains. Tabular Prior-Data Fitted Network (TabPFN), a recent foundation model for tabular data, has been shown capable of generating high-quality synthetic tabular data. However, TabPFN is autoregressive: features are generated sequentially by conditioning on the previous ones, depending on the order in which they appear in the input data. We demonstrate that when the feature order conflicts with causal structure, the model produces spurious correlations that impair its ability to generate synthetic data and preserve causal effects. We address this limitation by integrating causal structure into TabPFN's generation process through two complementary approaches: Directed Acyclic Graph (DAG)-aware conditioning, which samples each variable given its causal parents, and a partially directed acyclic graph (PDAG)-based strategy for scenarios with partial causal knowledge. We evaluate these approaches on controlled benchmarks and six CSuite datasets, assessing structural fidelity, distributional quality, and Average Treatment Effect (ATE) preservation. Across most settings, DAG-aware conditioning improves the quality and stability of synthetic data relative to vanilla TabPFN. Under partial causal knowledge, the oracle partially directed acyclic graph (oracle-PDAG), which orients only the edges into the colliders, shows moderate gains, while the benefit of a Completed Partially Directed Acyclic Graph (CPDAG) discovered from data depends on how well the causal structure is recovered. These results indicate that reliable causal structure, even partial, can be injected into TabPFN at inference time, without parameter updates, to improve synthetic data quality.

Davide Tugnoli, Andrea De Lorenzo, Marco Virgolin +1

Department of Mathematics, Informatics and Geosciences, University of Trieste, Trieste, Italy · InSilicoTrials Technologies, Riva Grumula 2, 34123 Trieste, Italy · Department of Engineering and Architecture, University of Trieste, Trieste, Italy +3

Synthetic Tabular Data Tabular Prior-Data Fitted Network Synthetic Data Directed Acyclic Graph

Mar 9, 2026cs.CV

Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning

This paper introduces a benchmark for evaluating whether vision-language models (VLMs) can generate plant simulation configurations from imagery using in-context learning. We study this benchmark for cowpea plot reconstruction for plant simulations, where the VLM needs to generate structured JSON configurations that include field and plant information. Open-source multimodal models from Gemma 4 and Qwen3.5 families are evaluated on a synthetic cowpea dataset with known JSON ground truth and on a real drone orthophoto dataset with field-collected JSON. Five in-context learning methods are used, from format restriction instruction to few-shot image examples with auxiliary grounding information. The results show that VLMs can generate valid JSON outputs, can generally estimate days after planting (DAP), plant counts, plant locations, sun angles, and leaf chlorophyll content, and can render approximate simulations of cowpea plots. Error metrics fluctuate across model families and often remain worse than dataset baselines, particularly when VLMs' pretrained knowledge dominates over weak visual evidence. These results position image-to-simulation JSON generation as a promising but currently challenging task, and establish a benchmark for studying how multimodal reasoning, prompt design, and the sim-to-real domain gap affect plant phenotyping tasks.

Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario +4

University of California, Davis

Vision-Language Foundation Models Visual Evidence Keyed Javascript Object Notation Record Yields In-Context Learning Synthetic Data Foundation Model

Feb 26, 2026astro-ph.IM

Reconstruction of cosmic-ray direction and energy in radio arrays using deep ensemble graph neural networks

Using advanced machine learning techniques, we developed a method to reconstruct the arrival direction and energy of ultra-high-energy cosmic rays from the voltage traces they induce on ground-based radio detector arrays. In our approach, triggered antennas are represented as a graph structure, which serves as input for a graph neural network (GNN). By incorporating physical knowledge into both the GNN architecture and the input data, we improve the precision and reduce the required size of the training set with respect to a fully data-driven approach. This method achieves an angular resolution of 0.092 degrees and an electromagnetic energy reconstruction resolution of 16.4% on simulated data with realistic noise conditions. We also employ uncertainty estimation methods to enhance the reliability of our predictions, quantifying the confidence of the GNN's outputs and providing confidence intervals for both direction and energy reconstruction. Finally, we investigate strategies to verify the model's consistency and robustness under real-life variations, with the goal of identifying scenarios in which predictions remain reliable despite domain shifts between simulation and reality.

Arsène Ferrière, Aurélien Benoit-Lévy, Olivier Martineau-Huynh +1

Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France · Sorbonne Université, CNRS, Laboratoire de Physique Nucléaire et des Hautes Énergies (LPNHE), 4 Pl. Jussieu, 75005 Paris, France · National Astronomical Observatories, Chinese Academy of Sciences, Beijing 100101, China +2

Cosmic-Ray Syntactic Pool Graph Neural Networks Synthetic Data

Feb 26, 2026cs.CL

MAPLE: Metadata Augmented Private Language Evolution

Differentially private (DP) fine-tuning of large language models (LLMs) requires massive compute and full model access, which rules out state-of-the-art proprietary APIs for general users. Generating DP synthetic data offers a practical workaround. This approach also allows for transparent exploratory data analysis and arbitrary reuse across downstream tasks, sidestepping the rigid constraints of a model's parameter space. Private Evolution (PE) provides a promising API-based framework for generating this data, but its success relies heavily on initialization. If the private data distribution falls too far outside the foundation model's pre-training priors -- a common issue in highly specialized domain -- PE struggles to align with the target data. This misalignment causes poor convergence, degraded utility, and wasted API calls. To solve this initialization bottleneck, we introduce Metadata Augmented Private Language Evolution (MAPLE). MAPLE extracts DP tabular metadata and uses in-context learning to firmly ground the initial synthetic distribution in the target domain. Our evaluations on domain-specific text generation tasks show that MAPLE yields a strictly better privacy-utility trade-off, converges significantly faster, and sharply reduces API costs compared to baseline PE methods.

Eli Chien, Yuzheng Hu, Ryan McKenna +3

National Taiwan University · University of Illinois Urbana-Champaign · Google Research +1

Synthetic Data Alphaevolve Evolutionary Optimization Methods Bottlenecks

Feb 16, 2026cs.LG

SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data

Improving Sparse Autoencoders (SAEs) requires benchmarks that can precisely validate architectural innovations. Current LLM-based SAE benchmarks are too noisy to differentiate architectural improvements, while commonly used synthetic-data experiments are too small-scale, unstandardized, and unrealistic to be meaningful. We introduce SynthSAEBench, a benchmark and toolkit for evaluating SAEs against large-scale synthetic data with realistic feature characteristics including correlation, hierarchy, and superposition, while providing ground-truth features and firings. SynthSAEBench acts as a controlled lower-bound test: SAE architectures that fail when the Linear Representation Hypothesis holds by construction have little hope on real LLMs. The benchmark reproduces known LLM SAE phenomena including the disconnect between reconstruction and latent quality, poor SAE probing, and a precision-recall trade-off mediated by L0, demonstrating that SynthSAEBench findings reproduce results on LLM SAEs. We further identify a novel failure mode: Matching Pursuit SAEs exploit superposition noise to improve reconstruction without learning ground-truth features, suggesting more expressive encoding procedures can easily overfit. SynthSAEBench complements LLM benchmarks with ground-truth features and controlled ablations for diagnosing SAE failure modes, while providing a clear target for SAE architecture work.

David Chanin, Adrià Garriga-Alonso

University College London · 2MATS · 3Decode Research

Sparse Autoencoder Features Improving Sparse Autoencoders Synthetic Benchmark Synthetic Data Large Language Model Benchmarks

Feb 10, 2026cs.CY

Stochastic Parrots or Singing in Harmony? Testing Five Leading LLMs for their Ability to Replicate a Human Survey with Synthetic Data

How well can AI-derived synthetic research data replicate the responses of human participants? An emerging literature has begun to engage with this question, which carries deep implications for organizational research practice. This article presents a comparison between a human-respondent survey of 420 Silicon Valley coders and developers and synthetic survey data designed to simulate real survey takers generated by five leading Generative AI Large Language Models: ChatGPT Thinking 5 Pro, Claude Sonnet 4.5 Pro plus Claude CoWork 1.123, Gemini Advanced 2.5 Pro, Incredible 1.0, and DeepSeek 3.2. Our findings reveal that while AI agents produced technically plausible results that lean more towards replicability and harmonization than assumed, none were able to capture the counterintuitive insights that made the human survey valuable. Moreover, deviations grouped together for all models, leaving the real data as the outlier. Our key finding is that while leading LLMs are increasingly being used to scale, replicate and replace human survey responses in research, these advances only show an increased capacity to parrot conventional wisdom in harmony with each other rather than revealing novel findings. If synthetic respondents are used in future research, we need more replicable validation protocols and reporting standards for when and where synthetic survey data can be used responsibly, a gap that this paper fills. Our results suggest that synthetic survey responses cannot meaningfully model real human social beliefs within organizations, particularly in contexts lacking previously documented evidence. We conclude that synthetic survey-based research should be cast not as a substitute for rigorous survey methods, but as an increasingly reliable pre- or post-fieldwork instrument for identifying societal assumptions, conventional wisdoms, and other expectations about research populations.

Jason Miklian, Kristian Hoelscher, John E. Katsos

University of Oslo · Peace Research Institute Oslo · American University of Sharjah

Survey Synthetic Data Generative Artificial Intelligence Large Language Model Responses Reproducibility Bird

Feb 5, 2026cs.CV

Efficient Dataset Distillation for Pre-Trained Self-Supervised Models via Statistical Flow Matching

Dataset distillation seeks to synthesize a compact surrogate dataset that enables performance comparable to training on the original dataset for downstream tasks. For the scenario where pre-trained self-supervised models serve as priors, traditional Linear Gradient Matching optimizes synthetic images by encouraging them to mimic the gradient updates induced by real images on the linear probe. However, this batch-level formulation requires loading thousands of real images and applying multiple differentiable augmentations to synthetic images at each distillation step, leading to substantial computational and memory overheads. In this paper, we revisit the linear gradient and theoretically derive that it is essentially a local relative distribution directed from target class centers toward non-target class centers, which we term flow. This property causes suboptimality and instability, often necessitating expensive multiple augmentations to compensate. To address this, we introduce Statistical Flow Matching, an optimal, stable, and efficient supervised learning framework that optimizes synthetic images by aligning global statistical flows in the original data. Our approach loads raw statistics only once and performs a single augmentation pass on the synthetic data, achieving performance comparable to or better than the state-of-the-art method with 10x less GPU memory usage and 4x faster distillation time. Moreover, increasing the number of augmentations for our method yields further performance gains while incurring lower additional cost. Our code is publicly available at https://github.com/einsteinxia/SFM.

Qianxin Xia, Jiawei Du, Yuhan Zhang +6

University of Electronic Science and Technology of China, Chengdu, China · CFAR, Singapore · SWJTU, Chengdu, China

Diffusion-Based Dataset Distillation Self-Supervised Learning Contrastive Flow Matching Conditional Flow Matching Synthetic Data Bilevel Optimization

Feb 4, 2026cs.CL

Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale

Contemporary studies in mechanistic interpretability have uncovered many puzzling phenomena in the neural information processing of Transformer-based language models, such as induction heads, function vectors, and the Hydra effect. Some of these individual phenomena have been independently tied to different data distributional properties, while some have been loosely associated with model architecture and how Transformers process information. However, a unified understanding of the relationship between data, model architecture, and optimization remains lacking, failing to answer the fundamental question: why do these three phenomena appear universally across different model families and scales, despite their seeming disconnect? In this work, we answer this question by unifying these three phenomena as consequences of hierarchical latent structures in the data generation process, coupled with decorrelated gradients across additive model components and directional concavity in the representation geometry. We validate our theoretical results in a toy model regime and in a large-scale synthetic data regime, comparing them with language models trained on natural language data.

Jonas Rohweder, Subhabrata Dutta, Iryna Gurevych

UKP Lab, TU Darmstadt

Mechanistic Interpretability Generalized Linear Model Synthetic Data

Feb 3, 2026cs.DB

PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models

Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train such models are rarely public due to privacy constraints. While there are methods to generate synthetic tabular data of arbitrary size, incorporating schema structure and primary-foreign key connectivity for multi-table generation remains challenging. Here we introduce PLUREL, a framework to synthesize multi-tabular relational databases from scratch. In a step-by-step fashion, PLUREL models (1) schemas with directed graphs, (2) inter-table primary-foreign key connectivity with bipartite graphs, and, (3) feature distributions in tables via conditional causal mechanisms. The design space across these stages supports the synthesis of a wide range of diverse databases, while being computationally lightweight. Using PLUREL, we observe for the first time that (1) RFM pretraining loss exhibits power-law scaling with the number of synthetic databases and total pretraining tokens, (2) scaling the number of synthetic databases improves generalization to real databases, and (3) synthetic pretraining yields strong base models for continued pretraining on real databases. Overall, our framework and results position synthetic data scaling as a promising paradigm for RFMs.

Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik +4

1Stanford University · 2Kumo AI · 3SAP Labs LLC

Relational Databases Synthetic Data Foundation Model Scaling Laws

Jan 30, 2026cs.CR

Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective

Training generative machine learning models to produce synthetic tabular data has become a popular approach for enhancing privacy in data sharing. As this typically involves processing sensitive personal information, releasing either the trained model or generated synthetic datasets can still pose privacy risks. Yet, recent research, commercial deployments, and privacy regulations like the General Data Protection Regulation (GDPR) largely assess anonymity at the level of an individual dataset. In this paper, we rethink anonymity claims about synthetic data from a model-centric perspective, arguing that meaningful assessments must account for the underlying generative model and be grounded in state-of-the-art privacy attacks. This perspective better reflects real-world deployments, where trained models are often accessible for interaction or querying. We interpret the GDPR's definitions of personal data and anonymization under such access assumptions to identify the identifiability risks that must be mitigated and map them to privacy attacks across threat settings. We then argue that synthetic data techniques alone do not ensure sufficient anonymization. Finally, we compare the two mechanisms most commonly used with synthetic data -- Differential Privacy (DP) and Similarity-based Privacy Metrics (SBPMs) -- and argue that while DP can offer robust protections against identifiability risks, SBPMs lack adequate safeguards. Overall, our work connects regulatory notions of identifiability with model-centric privacy attacks, enabling more responsible and trustworthy assessment of synthetic data systems by researchers, practitioners, and policymakers.

Georgi Ganev, Emiliano De Cristofaro

UCL · SAS · UC Riverside

Synthetic Data Anonymization Identifiability

Jan 5, 2026cs.RO

Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot

The development of robust and generalizable robot learning models is critically contingent upon the availability of large-scale, diverse training data and reliable evaluation benchmarks. Collecting data in the physical world poses prohibitive costs and scalability challenges, and prevailing simulation benchmarks frequently suffer from fragmentation, narrow scope, or insufficient fidelity to enable effective sim-to-real transfer. To address these challenges, we introduce Genie Sim 3.0, a unified simulation platform for robotic manipulation. We present Genie Sim Generator, a large language model (LLM)-powered tool that constructs high-fidelity scenes from natural language instructions. Its principal strength resides in rapid and multi-dimensional generalization, facilitating the synthesis of diverse environments to support scalable data collection and robust policy evaluation. We introduce the first benchmark that pioneers the application of LLM for automated evaluation. It leverages LLM to mass-generate evaluation scenarios and employs Vision-Language Model (VLM) to establish an automated assessment pipeline. We also release an open-source dataset comprising more than 10,000 hours of synthetic data across over 200 tasks. Through systematic experimentation, we validate the robust zero-shot sim-to-real transfer capability of our open-source dataset, demonstrating that synthetic data can server as an effective substitute for real-world data under controlled conditions for scalable policy training. For code and dataset details, please refer to: https://github.com/AgibotTech/genie_sim.

Chenghao Yin, Da Huang, Di Yang +16

Sim-To-Real Reinforcement Learning Scalable Robot Learning Synthetic Data High-Fidelity

Dec 2, 2025cs.LG

Training Energy-Based Models with Non-MCMC Samplers and Efficient Temperature Estimation

Efficient sampling from Boltzmann distributions over discrete variables is a fundamental operation in a wide range of applications. While fast non-MCMC samplers have recently emerged as promising alternatives to conventional MCMC methods, their practical use for probabilistic learning remains hindered by the difficulty of estimating the effective temperature of the generated samples. In this work, we begin by introducing Langevin simulated bifurcation (LSB), a Boltzmann sampler that enables fast and parallel sampling with accuracy comparable to sequential MCMC methods. To address the challenge of unknown effective temperature, we propose conditional expectation matching (CEM), an efficient estimation method applicable to energy-based models (EBMs) with exploitable conditional independence structures. Building on these components, we further develop a learning framework, termed sampler adaptive learning (SAL), which adaptively adjusts the model temperature to match that of the distribution induced by fast non-MCMC sampling. We demonstrate the effectiveness of LSB, CEM, and SAL on semi-restricted Boltzmann machines (SRBMs), a class of EBMs that are difficult to train using conventional approaches. LSB achieves orders-of-magnitude acceleration over Gibbs sampling while maintaining comparable or higher accuracy, and CEM enables accurate temperature estimation of the resulting distribution with negligible computational overhead. As a consequence, SAL enables efficient training of SRBMs and outperforms conventional Boltzmann machine learning methods on synthetic spin-glass datasets. In addition, the trained models achieve strong performance across multiple tasks. These results establish LSB as a fast and accurate Boltzmann sampler and provide key insights that enable practical applications of fast non-MCMC sampling methods via efficient temperature estimation with CEM.

Kentaro Kubo, Hayato Goto

Corporate Laboratory, Toshiba Corporation, Kawasaki, Kanagawa 212-8582, Japan · RIKEN Center for Quantum Computing (RQC), Wako, Saitama 351-0198, Japan

Markovian Sampling Markov Chain Monte Carlo Synthetic Data

Oct 21, 2025cs.LG

Learning Peer Influence Probabilities with Linear Contextual Bandits

In networked environments, it is common for users to share recommendations about content, products, services, and possible courses of action. Whether these recommendations are accepted and acted upon is highly context-dependent, influenced by the characteristics of the sender and recipient, the nature of their relationship, the attributes of the recommended item, and the communication context. Consequently, probabilities of peer influence exhibit substantial heterogeneity across individuals and settings. Accurate estimation of these probabilities is key to understanding information diffusion processes and to improving the effectiveness of viral marketing strategies. However, learning these probabilities from data is challenging; static data may capture correlations between peer recommendations and peer actions but fails to reveal influence relationships. Online learning algorithms can learn these probabilities from interventions but either waste resources by learning from random exploration or optimize for rewards, thus favoring exploration of the space with higher influence probabilities. In this work, we study learning peer influence probabilities under a contextual linear bandit framework. We show that a fundamental trade-off can arise between regret minimization and estimation error, characterize all achievable rate pairs, and propose an uncertainty-guided exploration algorithm that, by tuning a parameter, attains any pair within this trade-off. Our experiments on semi-synthetic network datasets show the advantages of our method over static methods and contextual bandits that ignore this trade-off.

Ahmed Sayeed Faruk, Mohammad Shahverdikondori, Elena Zheleva

University of Illinois Chicago, Chicago, USA · École Polytechnique Fédérale de Lausanne, Switzerland

Contextual Bandit Framework Influence Function Information-Flow Control Synthetic Data

Oct 18, 2025stat.ML

Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence

Synthetic data has been increasingly used to train frontier generative models. However, recent studies raise key concerns that iteratively retraining a generative model on its self-generated synthetic data may keep deteriorating model performance, a phenomenon often coined model collapse. In this paper, we investigate ways to modify the synthetic retraining process to avoid model collapse, and even possibly help reverse the trend from collapse to improvement. Our key finding is that by injecting information through an external synthetic data verifier, whether a human or a better model, synthetic retraining will not cause model collapse. Specifically, we situate our theoretical analysis in the fundamental linear regression setting, showing that verifier-guided retraining can yield near-term improvements, but ultimately drives the parameter estimate to the verifier's "knowledge center" in the long run. Our theory further predicts that, unless the verifier is perfectly reliable, these early gains will plateau and may even reverse. Indeed, our experiments across linear regression, Variational Autoencoders (VAEs) trained on MNIST, and fining-tuning SmolLM2-135M on the XSUM task confirm these theoretical insights.

Bingji Yi, Qiyuan Liu, Yuwei Cheng +1

Independent Researcher. Work done while visiting UChicago CS. · Work done while visiting UChicago CS. · Department of Statistics, University of Chicago. +1

Synthetic Data Retraining Collapse Generative Models Convergence

Oct 13, 2025cs.LG

Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation

Training student models on synthetic data generated by strong teacher models is a promising way to distilling the capabilities of teachers. However, recent studies show that stronger models are not always optimal teachers, revealing a mismatch between teacher outputs and student learnability. To address this issue, we propose PerSyn (Personalized data Synthesis), a novel synthesis strategy that operates under a new

Route then Generate'' paradigm to create data tailored to each student model, enabling it to learn more effectively. Specifically, PerSyn first assigns each prompt to its optimal teacher via a query-level router that jointly considers student learnability and teacher response quality. Each teacher then synthesizes data only for its assigned prompts, making the process more efficient than the conventional

Generate then Select'' paradigm, where all teachers must generate parallel responses for the entire prompt set before constructing the final dataset. Extensive experiments across different model families and scales demonstrate that PerSyn consistently achieves superior or comparable performance to all baselines in instruct tuning and math reasoning settings. Further analysis verifies the effectiveness of PerSyn and offers extra insights to propel future research.

Hengyuan Zhang, Shiping Yang, Xiao Liang +8

The University of Hong Kong · Simon Fraser University · University of California, Los Angeles +3

Teacher Synthetic Data

Oct 3, 2025cs.RO

MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning

Visual navigation policy is widely regarded as a promising direction, as it mimics humans by using egocentric visual observations for navigation. However, optical information of visual observations is difficult to be explicitly modeled like LiDAR point clouds or depth maps, which subsequently requires intelligent models and large-scale data. To this end, we propose to leverage the intelligence of the Vision-Language-Action (VLA) model to learn diverse navigation capabilities from synthetic expert data in a teacher-student manner. Specifically, we implement the VLA model, MM-Nav, as a multi-view VLA (with 360 observations) based on pretrained large language models and visual foundation models. For large-scale navigation data, we collect expert data from three reinforcement learning (RL) experts trained with privileged depth information in three challenging tailor-made environments for different navigation capabilities: reaching, squeezing, and avoiding. We iteratively train our VLA model using data collected online from RL experts, where the training ratio is dynamically balanced based on performance on individual capabilities. Through extensive experiments in synthetic environments, we demonstrate that our model achieves strong generalization capability. Moreover, we find that our student VLA model outperforms the RL teachers, demonstrating the synergistic effect of integrating multiple capabilities. Extensive real-world experiments further confirm the effectiveness of our method.

Tianyu Xu, Jiawei Chen, Jiazhao Zhang +5

Peking University, Beijing, China · Galbot, Beijing, China · Shanghai Jiao Tong University, Shanghai, China +1

Vision-Language Navigation Object Goal Navigation Diffusion-Based Vision-Language-Actions Menu Synthetic Data

Sep 25, 2025cs.DS

Actively Learning Halfspaces without Synthetic Data

In the classic point location problem, one is given an arbitrary dataset

X \subset \mathbb{R}^d

of

n

points with query access to an unknown halfspace

f : \mathbb{R}^d \to \{0,1\}

, and the goal is to learn the label of every point in

X

. This problem is extremely well-studied and a nearly-optimal

\widetilde{O}(d \log n)

query algorithm is known due to Hopkins-Kane-Lovett-Mahajan (FOCS 2020). However, their algorithm is granted the power to query arbitrary points outside of

X

(point synthesis), and in fact without this power there is an

Ω(n)

query lower bound due to Dasgupta (NeurIPS 2004). In this work our goal is to design efficient algorithms for learning halfspaces without point synthesis. To circumvent the

Ω(n)

lower bound, we consider learning halfspaces whose normal vectors come from a set of size

D

, and show tight bounds of

Θ(D + \log n)

. As a corollary, we obtain an optimal

O(d + \log n)

query deterministic learner for axis-aligned halfspaces, closing a previous gap of

O(d \log n)

vs.

Ω(d + \log n)

. In fact, our algorithm solves the more general problem of learning a Boolean function

f

over

n

elements which is monotone under at least one of

D

provided orderings. Our technical insight is to exploit the structure in these orderings to perform a binary search in parallel rather than considering each ordering sequentially, and we believe our approach may be of broader interest. Furthermore, we use our exact learning algorithm to obtain nearly optimal algorithms for PAC-learning. We show that

O(\min(D + \log(1/\varepsilon), 1/\varepsilon) \cdot \log D)

queries suffice to learn

f

within error

\varepsilon

, even in a setting when

f

can be adversarially corrupted on a

c\varepsilon

-fraction of points, for a sufficiently small constant

c

. This bound is optimal up to a

\log D

factor, including in the realizable setting.

Hadley Black, Kasper Green Larsen, Arya Mazumdar +2

Baruch College, CUNY · Aarhus University · University of California, San Diego

O(\Bar{K}\Log N)$Synthetic Data

Sep 13, 2025cs.CL

Privacy-Preserving Generation of Clinical Narratives from Medical Terminologies

In high-stakes domains such as healthcare, privacy concerns severely limit the use of real-world training data. Differentially private (DP) synthetic data offers a promising alternative with formal privacy guarantees, but achieving strong utility remains challenging for clinical note generation due to domain specificity and long-form text complexity. We present Term2Note, a method for synthesising full-length clinical notes under DP constraints. By structurally separating content and form, Term2Note generates section-wise note content conditioned on medical terms, with terms and notes privatised under separate DP constraints, and applies a DP quality maximiser to improve outputs. Experiments demonstrate that Term2Note produces synthetic notes with statistical properties closely aligned with real clinical notes, and that downstream models trained on these notes achieve performance comparable to those trained on real clinical data. Compared to existing DP text generation baselines, Term2Note substantially improves both fidelity and utility, without relying on label distribution assumptions, highlighting its effectiveness as a practical privacy-preserving alternative to real clinical notes.

Yuping Wu, Viktor Schlegel, Warren Del-Pinto +11

Univeristy of Manchester · Imperial College London, Imperial Global Singapore · Khoo Teck Puat Hospital, Singapore +2

Clinical Notes Privacy-Preserving Framework Synthetic Data Notes Narratives

Sep 12, 2025cs.LG

A Discrepancy-Based Perspective on Dataset Condensation

Given a dataset of finitely many elements

\mathcal{T} = \{\mathbf{x}_i\}_{i = 1}^N

, the goal of dataset condensation (DC) is to construct a synthetic dataset

\mathcal{S} = \{\tilde{\mathbf{x}}_j\}_{j = 1}^M

which is significantly smaller (

M \ll N

) such that a model trained from scratch on

\mathcal{S}

achieves comparable or even superior generalization performance to a model trained on

\mathcal{T}

. Recent advances in DC reveal a close connection to the problem of approximating the data distribution represented by

\mathcal{T}

with a reduced set of points. In this work, we present a unified framework that encompasses existing DC methods and extend the task-specific notion of DC to a more general and formal definition using notions of discrepancy, which quantify the distance between probability distribution in different regimes. Our framework broadens the objective of DC beyond generalization, accommodating additional objectives such as robustness, privacy, and other desirable properties.

Tong Chen, Raghavendra Selvan

Department of Computer Science, University of Copenhagen, Denmark

Diffusion-Based Dataset Distillation Distributional Fidelity Discrepancy Synthetic Data Discriminative Models

Sep 2, 2025stat.ML

Synthetic data for ratemaking: imputation-based methods vs adversarial networks and autoencoders

Actuarial ratemaking depends on high-quality data, yet access to such data is often limited by the cost of obtaining new data, privacy concerns, etc. In this paper, we explore synthetic-data generation as a potential solution to these issues. In addition to generative methods previously studied in the actuarial literature, we explore and benchmark another class of approaches based on Multivariate Imputation by Chained Equations (MICE). In a comparative study using an open-source dataset, MICE-based models are evaluated against other generative models like Variational Autoencoders and Conditional Tabular Generative Adversarial Networks. We assess how well synthetic data preserves the original marginal distributions of variables as well as the multivariate relationships among covariates. The consistency between Generalized Linear Models (GLMs) trained on synthetic data with GLMs trained on the original data is also investigated. Furthermore, we assess the ease of use of each generative approach and study the impact of generically augmenting original data with synthetic data on the estimation of GLMs for predicting claim counts. Our results highlight the potential of MICE-based methods in creating high-fidelity tabular data while offering lower implementation complexity compared to deep generative models.

Yevhen Havrylenko, Meelis Käärik, Artur Tuttar

Department of Actuarial Science, Faculty of Business and Economics, University of Lausanne, Switzerland · Institute of Mathematics and Statistics, Faculty of Science and Technology, University of Tartu, Estonia

Synthetic Data Actuarial Ratemaking Imputation Autoencoder Architectures Adversarial Training

Jul 21, 2025cs.AI

RAD: Retrieval High-quality Demonstrations to Enhance Decision-making

Offline reinforcement learning (RL) learns policies from fixed datasets, thereby avoiding costly or unsafe environment interactions. However, its reliance on finite static datasets inherently restricts the ability to generalize beyond the training distribution. Prior solutions based on synthetic data augmentation often fail to generalize to unseen scenarios in the (augmented) dataset. To address these challenges, we propose Retrieval High-quAlity Demonstrations (RAD) for decision-making, which innovatively introduces a retrieval mechanism into offline RL. Specifically, RAD retrieves high-return and reachable states from the offline dataset as target states, and leverages a generative model to generate sub-trajectories conditioned on these targets for planning. Since the targets are high-return states, once the agent reaches such a target, it can continue to obtain high returns by following the associated high-return actions, thereby improving policy generalization. Extensive experiments confirm that RAD achieves competitive or superior performance compared to baselines across diverse benchmarks, validating its effectiveness. Our code is available at https://anonymous.4open.science/r/RAD_0925_1-690E.

Lu Guo, Yixiang Shan, Zhengbang Zhu +5

School of Artificial Intelligence, Jilin University, Changchun, China · School of Computer Science, Shanghai Jiao Tong University, Shanghai, China

Model-Based Reinforcement Learning Offline Reinforcement Learning Improved Generalization Decisions Synthetic Data

Jun 11, 2025cs.CL

ToxSyn-PT: A Synthetic Fine-Grained Dataset of Minority-Targeted Toxic Language in Portuguese

The development of robust hate speech detection systems remains limited by the lack of large-scale, fine-grained training data, especially for languages beyond English. Existing corpora typically rely on simplistic toxic and non-toxic labels, and the few that capture hate directed at specific minority groups lack the positive counterexamples required to distinguish genuine hate from mere discussion. In this work, we introduce ToxSyn-PT, the first Portuguese large-scale corpus explicitly designed for multi-label hate speech detection across nine protected minority groups, including the non-toxic counterexamples absent in all other public datasets. Generated via a controllable four-stage pipeline, ToxSyn contains discourse-type annotations to capture rhetorical strategies of toxic/non-toxic language, such as sarcasm, dehumanization, and cultural appreciation. Our experiments reveal a catastrophic, mutual generalization failure compared to existing datasets from social-media domains: models trained on social media struggle to generalize to minority-specific contexts, and vice-versa. This finding indicates they are distinct tasks and exposes summary metrics like Macro F1 can be unreliable indicators of true model behavior, as they completely mask model failure. We publicly release ToxSyn on HuggingFace to support reproducible research on synthetic data generation and benchmark progress in hate-speech detection for low- and mid-resource languages.

Iago Alves Brito, Julia Soares Dollis, Fernanda Bufon Farber +2

Advanced Knowledge Center for Immersive Technologies (AKCIT) · Federal University of Goiás

Hate Speech Detection Natural Language Datasets Toxicity Minority Synthetic Data Portuguese

Jun 11, 2025cs.CV

ScoreMix: Synthetic Data Generation by Score Composition in Diffusion Models Improves Recognition

Synthetic data generation is increasingly used in machine learning for training and data augmentation. Yet, current strategies often rely on external foundation models or datasets, whose usage is restricted in many scenarios due to policy or legal constraints. We propose ScoreMix, a self-contained synthetic generation method to produce hard synthetic samples for recognition tasks by leveraging the score compositionality of diffusion models. The approach mixes class-conditioned scores along reverse diffusion trajectories, yielding domain-specific data augmentation without external resources. We systematically study class-selection strategies and find that mixing classes distant in the discriminator's embedding space yields larger gains, providing up to 3% additional average improvement, compared to selection based on proximity. Interestingly, we observe that condition and embedding spaces are largely uncorrelated under standard alignment metrics, and the generator's condition space has a negligible effect on downstream performance. Across 8 public face recognition benchmarks, ScoreMix improves accuracy by up to 7 percentage points, without hyperparameter search, highlighting both robustness and practicality. Our method provides a simple yet effective way to maximize discriminator performance using only the available dataset, without reliance on third-party resources. Paper website: https://parsa-ra.github.io/scoremix/.

Parsa Rahimi, Sebastien Marcel

1EPFL, Switzerland · 2Idiap, Switzerland

Synthetic Data Data Augmentation Diffusion Models

Mar 17, 2025cs.CV

Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs

Vision-Language Models (VLMs) have demonstrated exceptional performance in various multi-modal tasks. Recently, there has been an increasing interest in improving the personalization capabilities of VLMs. To better integrate user-provided concepts into VLMs, many methods use positive and negative samples to fine-tune these models. However, the scarcity of user-provided positive samples and the low quality of retrieved negative samples pose challenges for existing techniques. To reveal the relationship between sample and model performance, we systematically investigate the amount and diversity impact of positive and negative samples (easy and hard) on VLM personalization tasks. Based on the detailed analysis, we introduce Concept-as-Tree (CaT), which represents a concept as a tree structure, thereby enabling the data generation of positive and negative samples with varying difficulty and diversity, and can be easily extended to multi-concept scenarios. With a well-designed data filtering strategy, our CaT framework can ensure the quality of generated data, constituting a powerful pipeline. We perform thorough experiments with various VLM personalization baselines to assess the effectiveness of the pipeline, alleviating the lack of positive samples and the low quality of negative samples. Our results demonstrate that CaT equipped with the proposed data filter significantly enhances the capabilities of VLMs across personalization benchmarks. To the best of our knowledge, this work is the first controllable synthetic data pipeline for VLM personalization.

Ruichuan An, Kai Zeng, Ming Lu +5

1Peking University · 2CUHK MMLab · 3Zhongguancun Academy +1

Recent Vision-Language Models Synthetic Data Negative Results Diversity

Feb 27, 2025cs.LG

Sanity Checking Causal Representation Learning on a Simple Real-World System

We evaluate methods for causal representation learning (CRL) on a simple, real-world system that satisfies the basic problem setup of CRL. The system consists of a controlled optical experiment producing a variety of measurements where the underlying causal factors---the control inputs to the experiment---are known, providing a ground truth. We select methods representative of different approaches to CRL and find that they all fail to consistently recover the underlying causal factors. To understand the failure modes of the evaluated algorithms, we perform an ablation on the data by substituting the real data-generating process with a simpler synthetic equivalent. The results reveal a reproducibility problem, as most methods already fail on this synthetic ablation despite its simple data-generating process. Additionally, we observe that common assumptions on the mixing function are crucial for the performance of some of the methods but do not hold in the real data. Our efforts highlight the contrast between the theoretical promise of the state of the art and the challenges in its application. We hope the benchmark serves as a simple, real-world sanity check to further develop and validate methodol- ogy, bridging the gap towards CRL methods that work in practice.

Juan L. Gamella, Simon Bing, Jakob Runge

Seminar for Statistics, ETH Zurich · Technische Universität Berlin · Department of Computer Science, University of Potsdam +1

Causal Representation Learning Causal Synthetic Data Efficient Sequence Mixers Optical

Feb 25, 2025stat.ML

Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework

Recent studies identified an intriguing phenomenon in recursive generative model training known as model collapse, where models trained on data generated by previous models exhibit severe performance degradation. Addressing this issue and developing more effective training strategies have become central challenges in generative model research. In this paper, we investigate this phenomenon within a novel framework, where generative models are iteratively trained on a combination of newly collected real data and synthetic data from the previous training step. To develop an optimal training strategy for integrating real and synthetic data, we evaluate the performance of a weighted training scheme in various scenarios, including Gaussian distribution estimation, generalized linear models, and nonparametric estimation. We theoretically characterize the impact of the mixing proportion and weighting scheme of synthetic data on the final model's performance. Our key finding is that, across different settings, the optimal weighting scheme under different proportions of synthetic data asymptotically follows a unified expression, revealing a fundamental trade-off between leveraging synthetic data and model performance. In some cases, the optimal weight assigned to real data corresponds to the reciprocal of the golden ratio. Finally, we validate our theoretical results on extensive simulated datasets and a real tabular dataset.

Hengzhi He, Shirong Xu, Guang Cheng

Department of Statistics and Data Science, University of California, Los Angeles · Wang Yanan Institute for Studies in Economics, Xiamen University

Synthetic Data Generative Models Collapse Stabilization

Feb 21, 2025cs.LG

Mantis: Lightweight Foundation Model for Time Series Classification

While foundation models have revolutionized various domains, their application to time series classification remains rather under-explored, with existing literature predominantly focused on forecasting. To bridge this gap, we introduce \textbf{Mantis}, a transformer-based foundation model pre-trained exclusively on synthetic data via self-supervised contrastive learning. We demonstrate that effective tokenization is critical to unlocking the full potential of transformers, proposing a novel token generator unit. Furthermore, we introduce an enhanced test-time methodology that bridges the performance gap between Mantis and strong specialized approaches by leveraging intermediate-layer representations, self-ensembling, and cross-model embedding fusion. Extensive experiments demonstrate that Mantis establishes a new state-of-the-art, outperforming existing foundation models across four diverse dataset collections covering various application domains.

Vasilii Feofanov, Songkang Wen, Shifeng Xie +10

1. Huawei Noah’s Ark Lab · 3. Paris Cit´e University · 4. Helmholtz Munich +3

Time Series Foundation Models Time-Series Classification Wireless Foundation Models Contrastive Learning Synthetic Data

Dec 24, 2024cs.CL

Learning from Many Voices: Literary MT Using Multi-Reference Human and Synthetic Data

Unlike many other texts, literary works are often translated multiple times. We investigate strategies for leveraging these multi-reference datasets to improve literary machine translation. We propose a filtering framework based on semantic similarity to identify source texts whose references display meaningful variation while remaining faithful. We find that fine-tuning with medium to high semantic similarity data substantially outperforms low semantic similarity data. Moreover, using medium and high semantic similarity data achieves comparable or better performance than using the full unfiltered data. Synthetic translations generated by LLMs are economical and convenient alternatives to human expert translations; however, we find fine-tuning on human expert translations outperforms fine-tuning on synthetically augmented data in automatic metrics and human evaluations, demonstrating the indispensable value of human expert translations for fine-tuning literary machine translation models.

Si Wu, John Wieting, David A. Smith

Northeastern University · Google DeepMind

Machine Translation Multimodal Dataset Similarity and Metrics Synthetic Data

Nov 11, 2024eess.IV

Evaluating Synthetic Data Generation for Domain Generalization in Fetal Brain MRI Segmentation

Fetal brain tissue segmentation from magnetic resonance imaging (MRI) is crucial for studying neurodevelopment, but remains challenging due to data heterogeneity and limited annotations. Domain randomization (DR) has recently emerged as a promising strategy for single-source domain generalization by synthesizing training images with randomized artifacts, contrast, and resolution. In this work, we investigate how to maximize the out-of-domain (OOD) generalization of DR-based methods. We evaluate several synthetic data generation strategies for DR, with a particular focus on our recently proposed framework, FetalSynthSeg. We show that simple Gaussian mixture-based intensity modeling outperforms more complex physics-based simulations, and that intensity clustering (subdividing tissue classes based on intensity) improves OOD robustness. Evaluated on 348 fetal subjects from four sites spanning 0.55-3T and both T1w and T2w contrasts, FetalSynthSeg reaches state-of-the-art performance on several FeTA 2024 testing datasets (80-85 Dice score) and, for the first time, offers robust segmentation on modalities other than T2w for fetal brain segmentation (80 Dice on dHCP-T1w dataset). Compared with state-of-the-art methods such as BOUNTI, nnU-Net ensemble, and the FeTA 2024 winner, FetalSynthSeg delivers comparable or superior accuracy while maintaining strong robustness across domain shifts. Our code, model weights, and Docker image ready for easy inference are available at https://hub.docker.com/r/vzalevskyi/fetalsynthseg.

Vladyslav Zalevskyi, Thomas Sanchez, Margaux Roulet +10

Department of Radiology, Lausanne University Hospital and University of Lausanne (UNIL), Lausanne, Switzerland · CIBM Center for Biomedical Imaging, Lausanne, Switzerland · Institute for Information Processing, Leibniz University Hannover, Hannover, Germany +5

Medical Image Generation Magnetic Resonance Imaging Multimodal Domain Generalization Synthetic Data Random Mixture Models

Sep 27, 2024cs.LG

HardCore Generation: Generating Hard UNSAT Problems for Data Augmentation

Efficiently determining the satisfiability of a boolean equation -- known as the SAT problem for brevity -- is crucial in various industrial problems. Recently, the advent of deep learning methods has introduced significant potential for enhancing SAT solving. However, a major barrier to the advancement of this field has been the scarcity of large, realistic datasets. The majority of current public datasets are either randomly generated or extremely limited, containing only a few examples from unrelated problem families. These datasets are inadequate for meaningful training of deep learning methods. In light of this, researchers have started exploring generative techniques to create data that more accurately reflect SAT problems encountered in practical situations. These methods have so far suffered from either the inability to produce challenging SAT problems or time-scalability obstacles. In this paper we address both by identifying and manipulating the key contributors to a problem's ``hardness'', known as cores. Although some previous work has addressed cores, the time costs are unacceptably high due to the expense of traditional heuristic core detection techniques. We introduce a fast core detection procedure that uses a graph neural network. Our empirical results demonstrate that we can efficiently generate problems that remain hard to solve and retain key attributes of the original example problems. We show via experiment that the generated synthetic SAT problems can be used in a data augmentation setting to provide improved prediction of solver runtimes.

Joseph Cotnareanu, Zhanguang Zhang, Hui-Ling Zhen +2

McGill University Montreal, Canada · Huawei Noah’s Ark Lab Montreal, Canada · Huawei Noah’s Ark Lab Hong Kong, China +1

Satisfiability Synthetic Data

Aug 25, 2023eess.SY

From system models to class models: An in-context learning paradigm

Is it possible to understand the intricacies of a dynamical system not solely from its input/output pattern, but also by observing the behavior of other systems within the same class? This central question drives the study presented in this paper. In response to this query, we introduce a novel paradigm for system identification, addressing two primary tasks: one-step-ahead prediction and multi-step simulation. Unlike conventional methods, we do not directly estimate a model for the specific system. Instead, we learn a meta model that represents a class of dynamical systems. This meta model is trained on a potentially infinite stream of synthetic data, generated by simulators whose settings are randomly extracted from a probability distribution. When provided with a context from a new system-specifically, an input/output sequence-the meta model implicitly discerns its dynamics, enabling predictions of its behavior. The proposed approach harnesses the power of Transformers, renowned for their \emph{in-context learning} capabilities. For one-step prediction, a GPT-like decoder-only architecture is utilized, whereas the simulation problem employs an encoder-decoder structure. Initial experimental results affirmatively answer our foundational question, opening doors to fresh research avenues in system identification.

Marco Forgione, Filippo Pura, Dario Piga

IDSIA Dalle Molle Institute for Artificial Intelligence USI-SUPSI, Via la Santa 1, CH-6962 Lugano-Viganello, Switzerland.

System Identification In-Context Learning Next-State Prediction Meta-Learning Sequence Modeling Decoder Layers+1

Date pendingcs.LG

Assessing Predictive Models for Fairness Based on Activity-Space Patterns

Assessing the spatial fairness of predictive models involves establishing whether they are statistically penalizing (favoring) individuals associated with certain geographical locations. Literature on this topic makes the fundamental assumption that each individual is assigned to a single geographical location (e.g., place of residence). However, fairness with respect to the set of regions where one regularly spends time, i.e., the individual's activity space, also matters when fairness is considered. Consequently, we argue that it is necessary to generalize the notion of spatial fairness to also account for such activity-space patterns, leading to the novel problem of assessing predictive models for fairness relative to the movements of individuals. To deal with this problem, we propose an approach that first associates individuals with geographic regions relevant to their activity spaces, considering multiple spatial partitions with different resolutions and alignments, and then employs a suitable spatial scan statistic to assess whether a predictive model is fair based on activity-space patterns. In the experimental evaluation, we study the performance of our approach over thousands of synthetic unfair datasets, showing that it is effective at detecting this new type of unfairness and at retrieving the set of objects treated unfairly, while localization performance exhibits a consistent multi-resolution trade-off.

Francesco Lettich, Mario A. Nascimento, Chiara Pugliese +1

Institute of Information Science and Technologies, National Research Council, Pisa, Italy · Department of Computing Science, University of Alberta, Edmonton, Canada · Institute of Informatics and Telematics, National Research Council, Pisa, Italy

Algorithmic Fairness Predictability Heterogeneity Localization Synthetic Data Motion

Date pendingecon.EM

Synthetic Blips: Generalizing Synthetic Controls for Dynamic Treatment Effects

We propose a generalization of the synthetic control methods to the setting with dynamic treatment effects, in which each unit receives multiple treatments sequentially, according to an adaptive policy that depends on a latent, endogenously time-varying confounding state. Under a low-rank latent factor model assumption, which admits linear time-varying and time-invariant dynamic triangular systems as special cases, we develop an identification strategy for any unit-specific mean outcome under any sequence of interventions. Our method, which we term synthetic blips, is a backward induction process in which the blip effect of a treatment at each period for a target unit is recursively expressed as a linear combination of the blip effects of other units that received the designated treatment, avoiding the combinatorial donor requirements of naive synthetic control extensions. We provide easy-to-implement estimation algorithms that yield consistent estimators. Using unique Korean firm-level panel data, we estimate individualized dynamic treatment effects and optimal allocation rules in the context of financial support for exporting firms.

Anish Agarwal, Sukjin Han, Dwaipayan Saha +2

Columbia University · University of Bristol · Stanford University +1

Heterogeneous Treatment Effects Treatment Allocation Synthetic Data Treatment Backward Passes Impact Analysis

Date pendingeess.AS

ProPS: Prompted Profile Synthesis for Natural Language-Conditioned Speaker Embedding Distributions

Speaker embeddings, or x-vectors, are widely used to represent speaker identity and speaker-related attributes, but existing embedding extractors are typically descriptive rather than generative: they map an observed speech segment to an x-vector, which is then used for downstream applications. We introduce ProPS, Prompted Profile Synthesis, a framework for generating distributions of speaker embeddings conditioned on natural language prompts such as "a thirties male speaker with an Indian accent". ProPS converts human-written profile descriptions into sentence embeddings and uses a mixture density network trained on a large-scale dataset to predict a Gaussian mixture model in the x-vector space. The model is trained by maximizing the likelihood that real speaker embeddings match the requested profile, and its generated distributions are evaluated by negative log-likelihood on held-out x-vectors and by attribute classification accuracies on sampled synthetic x-vectors. Experiments show that ProPS produces profile-conditioned distributions and generates x-vectors that preserve requested speaker attributes such as age, gender, accent, and prosodic characteristics. This design enables controllable speaker-profile synthesis for speech generation systems like Text-To-Speech (TTS) or Voice Conversion (VC) while anchoring generated distributions in observed speaker-embedding structure.

Thomas Thebaud, Junhyeok Lee, Laureano Moro-Velazquez +2

Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA · AMIAD, Palaiseau, France · Department of Signal Theory and Communications, Universidad Carlos III de Madrid (UC3M), Leganes, Spain

Speech Synthesis Speaker Profile Synthetic Data Distributions

Date pendingcs.AI

PCBWorld: A Benchmark Environment for Engine-Grounded PCB Design Automation

PCB routing is the task of connecting the nets of a board with copper traces under strict design rules, yet learning-based methods still lag behind rule-based routers. We introduce PCBWorld, an open-source engine-grounded PCB routing environment built on KiCad, an electronic design automation (EDA) engine. As a human engineer does, agents in PCBWorld interactively route a board through the engine's native operations, guided by its Design Rule Check (DRC) feedback. The environment supports both RL and tool-using LLM agents. Alongside the environment, PCBWorld-Bench provides three board datasets in the native .kicad_pcb format, two controllable synthetic generators and 679 real open-source boards. It scores any completed board with eight engine-checked evaluation metrics, regardless of the routing method. In our experiments, agents in PCBWorld consistently outperformed grid-action RL policies and open-loop LLM baselines, and an RL policy trained only on synthetic boards transferred zero-shot to real boards, approaching rule-based routers. PCBWorld and PCBWorld-Bench are available at https://github.com/LGAI-Research/PCBWorld.

Hyungseok Song, Junseok Park, Won-Seok Choi +4

LG AI Research Seoul, Republic of Korea

Printed Circuit Board Icr-Rl Synthetic Data

Date pendingcs.CL

A Factorial Study of Synthetic Data Generation for Low-Resource Machine Translation using Grammar Books

Most endangered languages lack the parallel data required for machine translation, despite the existence of descriptive grammar books. We introduce a pipeline that uses large language models to extract grammatical rules, example sentences, and lexicons from grammar books and generate synthetic parallel corpora for fine-tuning-rather than feeding grammar content into prompts at inference time, as in prior work. Validated on three typologically diverse low-resource languages-Kalamang (Papuan), Tuatschin (Romance), and Mandan (Siouan)-we show that fine-tuning on synthetic data improves over seed-data baselines in 75% of configurations for Kalamang and 59% for Tuatschin, with best-case ChrF++ gains of +8.8, +5.3, and +3.3 respectively. Through a systematic factorial study across 96 configurations varying target part-of-speech, retrieval granularity, and sample volume, we identify which factor combinations drive gains and where they break down. Our results demonstrate that static linguistic documentation can be repurposed for machine translation fine-tuning, offering a practical path towards translation tools for severely under-resourced languages.

Varun Ghat Ravikumar, Sina Ahmadi, Lena Jäger +1

Department of Informatics, University of Zurich · Department of Computational Linguistics, University of Zurich

Machine Translation Low-Resource Languages Text Corpora Synthetic Data Factor

Date pendingstat.ML

Bias-Corrected Data Synthesis for Imbalanced Learning

Class imbalance complicates probabilistic classification because standard training objectives emphasize majority-class performance. Synthetic oversampling can reduce imbalance, but discrepancies between the synthetic and target minority distributions may bias the fitted classifier, especially because synthetic samples depend on the observed data. We propose a bias-correction procedure that estimates the generator-induced loss discrepancy from a held-out subset of majority observations and transfers this correction to the minority class under a uniform bias-transfer condition. We establish finite-sample bounds for bias transfer and for the excess balanced risk of the resulting empirical risk minimizer, and characterize a regime in which SMOTE induces non-negligible loss bias. The framework can also be implemented in imbalanced multi-task learning and propensity-score estimation, with details provided in the Supplementary Material. Real data analyses show that the correction is most useful when synthetic distortion is appreciable.

Pengfei Lyu, Zhengchi Ma, Linjun Zhang +1

Department of Biostatistics & Bioinformatics, Duke University · Department of Electrical & Computer Engineering, Duke University · Department of Statistics, Rutgers University +1

Oversampling Class Imbalance Synthetic Data Loss Function Majority Voting