Learning a Size-Weight Frontier for Synthetic-Augmented Inference

Chengpiao Huang; Kaizheng Wang

stat.MEAug 28, 2026

Learning a Size-Weight Frontier for Synthetic-Augmented Inference

Authors: Chengpiao Huang, Kaizheng Wang

Organizations: Department of IEOR, Columbia University · Department of IEOR and Data Science Institute, Columbia University

Abstract

Synthetic data can improve statistical inference when real data are scarce, but naively treating synthetic samples as real data can introduce bias and lead to unreliable inference. We develop a general framework for synthetic-augmented inference across a population of related tasks. It characterizes synthetic augmentation by the number of synthetic observations and their weight. Central to our framework is a size-weight frontier that specifies, for each weight, the largest synthetic sample size for which all smaller sizes attain the target task-marginal coverage. We estimate this frontier from historical tasks, and establish a finite-sample coverage guarantee simultaneously for all size-weight configurations on or below the estimated frontier. In experiments using large language model responses to augment opinion survey data, our procedure achieves target coverage and substantially narrows confidence intervals.

Figures & tables

Figure 1: Illustration of a Size–Weight Frontier

Figure 2: Learned and Oracle Size–Weight Frontiers for GPT-4o

Figure 3: Empirical Test Coverage

\widehat{p}_{n,\mathrm{te}}(\lambda,\widehat{k}_{n}(\lambda))

at the Learned Frontier

Figure 4: Average Relative Confidence Interval Width Reduction

\overline{R}_{n}(\lambda)

Figure 5: Effective Synthetic Sample Size

\lambda\widehat{k}_{n}(\lambda)

at the Learned Frontier

Synthetic Data Augmentation

Synthetic data augmentation creates or modifies training examples to improve learning when real data are scarce, imbalanced, costly, or sensitive.

See all 454 papers

Valid Inference with Synthetic Data via Task Exchangeability
Jun 11, 2026Lezhi Tan, Tijana ZrnicSynthetic Data Evaluation
Department of Management Science & Engineering Stanford University · Department of Statistics Stanford University
Training-Aware Target Coverage for Synthetic Data Selection
Sep 30, 2026Yang Ba, Michelle V. Mancenido, Rong PanLLM Fine-Tuning Data Selection
School of Computing and Augmented Intelligence, Arizona State University · School of Mathematical and Natural Sciences, Arizona State University
Improving Machine Learning Performance with Synthetic Augmentation
Apr 16, 2026Mel Sohm, Charles Dezons, Sami Sellami +2Synthetic Data Augmentation Quantitative Finance
University of California, Berkeley

Jun 11, 2026J/K move · Enter open · S save

Valid Inference with Synthetic Data via Task Exchangeability

Lezhi Tan, Tijana Zrnic

Department of Management Science & Engineering Stanford University · Department of Statistics Stanford University

Synthetic Data Evaluation

Open paper AskPDF arXiv

There is a proliferation of work arguing for the use of synthetic data in scientific research. For example, social scientists are arguing for the use of LLM-generated "silicon samples" in pilot studies; AI evaluations increasingly rely on "LLM-as-a-judge" outputs; and proteomics research is accelerated by generative models that produce synthetic protein structures. These developments raise an intriguing possibility: synthetic data may help researchers ask more questions, run more studies, and accelerate discovery. But they also raise a fundamental concern: synthetic data can be biased, noisy, and misspecified. In this work, we propose statistical principles for using synthetic data in scientific research with provable validity guarantees. The key insight is a new technical condition that we call task exchangeability. Informally, this is a requirement that the researcher can identify historical tasks, for which real data is available, such that their current task of interest is exchangeable with the historical tasks in an appropriate mathematical sense. We develop methods for valid inference under task exchangeability, together with extensions that provide guarantees even beyond exchangeability. We demonstrate the framework on public opinion surveys with silicon samples and AI evaluation with autoraters.

Abstract

Synthetic Data Augmentation

Explore similar work

Valid Inference with Synthetic Data via Task Exchangeability

Training-Aware Target Coverage for Synthetic Data Selection

Improving Machine Learning Performance with Synthetic Augmentation