Costly simulations and experiments limit the data available for engineering design. This paper proposes GenPFN, a training-free candidate generator guided by a pretrained tabular Prior-Data Fitted Network (TabPFN) classifier with fixed weights. The context contains evaluated designs labeled desired (Class 1) or undesired (Class 0). GenPFN interpolates between desired designs and refines candidates using gated classifier gradients and Gaussian noise. Each candidate is evaluated using the problem's constraints and objective functions. Evaluated desired designs and their objective values augment the context for regression training. Evaluation spans fourteen selected problems, including a two-parameter benchmark and a 44-parameter hull problem. In paired comparisons, GenPFN with TabPFN v1 achieves higher class satisfaction and lower realism distance than the tested adversarial, variational-autoencoder and diffusion generators. The variational-autoencoder generator provides greater coverage and diversity. Relative to the synthetic minority over-sampling technique (SMOTE), GenPFN provides lower coverage distance and greater diversity. Among observed paired regression fits, augmentation reduces estimated AutoGluon validation mean squared error by 62.1% relative to context-only training. A hull-stern case study yields 99.3% propeller-compatible candidates versus 1.4% among randomly sampled context designs. Generation followed by evaluation produces sample sets that augment limited datasets and support training more accurate predictive models.
Figures & tables
Figure 1: GenPFN candidate generation. A pretrained TabPFN v1 classifier uses evaluated designs and their class labels as context while its weights remain fixed. Candidates begin between observed Class 1 designs. Gated classifier gradients direct their updates, gradient clipping limits the directed movement, and Gaussian noise adds variation. Arrows show classifier guidance before clipping and noise are applied. Blue markers represent proposed designs; subsequent problem-specific evaluation determines whether they satisfy the target requirements.
Figure 2: One-dimensional gradient-activation rules. The red curve shows the predicted Class 1 probability p(x) ; the blue dashed curves show (a) ∣p′(x)∣ , (b) the log-activated magnitude ∣p′(x)/(p(x)+δ)∣ , and (c) the ReLU-activated magnitude used by GenPFN. The graphic omits δ=10−6 because it is not visible at this scale, but Equations 7 and 10 retain it. The ReLU-activated field equals the log-activated field below P=0.5 and is zero otherwise. Both fields precede the transformation in Equation 4 .
Figure 3: TabPFN fields for the Two Circles problem: (a) in-context samples and ground-truth classes, (b) predicted Class 1 probability, (c) the log-activated field gL , and (d) the ReLU-activated field gR , which is zero for pθ≥P=0.5 . The embedded activation labels abbreviate these fields. The fields are computed before Sλ and Gaussian noise, so they do not show final candidate displacements.
Figure 4: Required clearances for the 3.2-m-diameter propeller: a>32 mm, b>640 mm, c>960 mm, and d>1,600 mm.
Figure 5: Row-17 parent hull with a 0.1519m2 admissible placement area for the propeller-hub center.
Figure 6: Initial hull for the stern-design case study.
Figure 7: Candidate distributions for the Two Circles problem using one 64-design context with equal class proportions and subset 0: (a) GenPFN v1, (b) continuous log-activated guidance with direct initialization, (c) SMOTE, (d) CTGAN, (e) TVAE, and (f) TabDDPM. Blue markers denote proposals, while the background gives the ground-truth class regions. Axes show the original unit-square viewport; proposals outside this viewport remain part of the evaluated pool. CTGAN’s mean Class 1 rate of 17.8% summarizes the full Two Circles benchmark.
Figure 8: Sample-set metrics for Two Circles versus the number of Class 1 context samples: (a) feasibility, (b) realism Chamfer distance, (c) coverage Chamfer distance, and (d) mean nearest-neighbor distance. Solid and dashed blue lines distinguish GenPFN v1 and v3; the remaining methods retain the original colors. Higher feasibility and nearest-neighbor distance and lower realism and coverage distances are preferred. Lines show means; bands show pointwise 95% context-bootstrap intervals.
Figure 9: Sample-set metrics for Hull versus the number of Class 1 context samples, using the same panels, method colors, backbone line styles and pointwise 95% context-bootstrap intervals as Figure 8 .
Figure 10: Hull AutoGluon validation MSE versus the number of Class 1 context samples. Only externally evaluated feasible candidates augment the context. Missing fits remain gaps.
Figure 11: Two Circles AutoGluon validation MSE versus the number of Class 1 context samples. The context-only curve uses the same reference fits as the paired analysis.
Figure 12: Evaluated GenPFN stern candidate with a 1.079m2 propeller-hub placement area.
Figure 13: Overall hull for the candidate in Figure 12 ; only the stern parameters differ from the initial hull in Figure 6 .
Figure 14: Additional evaluated GenPFN stern candidates selected at the 10th, 50th and 90th percentiles of placement area among the 993 target-compatible generated designs. Only the permitted stern parameters differ from the initial hull.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 15: Two Circles initialization using 64 context designs with equal class proportions, subset 0, and σ=0.0005 . Direct starts above and interpolated starts below both use gated guidance.
Figure 16: Two Circles candidates generated using continuous and gated probability guidance from matching interpolated Class 1 starts. Both configurations use 64 context designs with equal class proportions and subset 0.
Figure 17: Heat Exchanger sample-set metrics for interpolated v1 gated and continuous guidance, noise-only interpolation and pure interpolation. Panels retain feasibility, realism, coverage and nearest-neighbor dispersion roles.
Figure 18: Heat Exchanger AutoGluon validation mean squared error for the interpolated controls and probability-guidance arms, with context-only training as reference. Curves summarize observed regression fits.
Figure 19: Gated GenPFN and noise-only feasibility trajectories using the stated sampling settings. Bands are pointwise paired-context bootstrap intervals; the 30,000-update endpoint was specified in advance.
Generative tabular augmentation is appealing in data-scarce domains, yet the prevailing focus on distributional fidelity does not reliably translate into better downstream models. We formalize a fidelity-utility gap: common generative objectives prioritize distributional plausibility, whereas augmentation succeeds only when injected samples reduce the current learner's held-out evaluation loss. This gap motivates learning not just how to generate, but what to generate and when to inject as training evolves. We propose TAP (Tabular Augmentation Policy), which couples diffusion inpainting with a lightweight, learner-conditioned policy to steer generation toward high-utility regions and controls safe injection via explicit gating and conservative windowed commitment. Under severe data scarcity, TAP consistently outperforms strong generative baselines on seven real-world datasets, improving classification accuracy by up to 15.6 percentage points and reducing regression RMSE by up to 32%.
Zheyu Zhang, Shuo Yang, Bardh Prenkaj +1
Technical University of Munich · Munich Center for Machine Learning (MCML)
Predictive modeling in engineering applications has long been dominated by bespoke models and small, siloed tabular datasets, limiting the applicability of large-scale learning approaches. Despite recent progress in tabular foundation models, the resulting synthetic training distributions used for pre-training may not reflect the statistical structure of engineering data, limiting transfer to engineering regression. We introduce TREDBench, a curated collection of 83 real-world tabular regression datasets with expert engineering/non-engineering labels, and use TabPFN 2.5's dataset-level embedding to study domain structure in a common representation space. We find that engineering datasets are partially distinguishable from non-engineering datasets, while standard procedurally generated datasets are highly distinguishable from engineering datasets, revealing a substantial synthetic-real domain gap. To bridge this gap without training on real engineering samples, we propose an embedding-guided synthetic data curation method: we generate and identify "engineering-like" synthetic datasets, and perform continued pre-training of TabPFN 2.5 using only the selected synthetic tasks. Across 35 engineering regression datasets, this synthetic-only adaptation improves predictive accuracy and data efficiency, outperforming TabPFN 2.5 on 29/35 datasets and AutoGluon on 27/35, with mean multiplicative data-efficiency gains of 1.75x and 4.44x, respectively. More broadly, our results indicate that principled synthetic data curation can convert procedural generators into domain-relevant "data engines," enabling foundation models to improve in data-sparse scientific and industrial domains where real data collection is the primary bottleneck.
Tabular data generation aims to synthesize realistic samples from limited observations while faithfully capturing the underlying data distribution. Tabular foundation models (TFMs), pretrained on diverse tabular tasks and adapted through in-context learning, have demonstrated strong predictive capabilities, suggesting the potential of extending this paradigm to tabular data generation. However, the strong predictive performance of TFMs often relies on sufficiently large context sets, whereas tabular generation is typically needed precisely when target data are scarce. Extending the foundation-model paradigm to generation therefore requires reliable distribution inference from only a limited context. To this end, we propose TabGift, a context-efficient tabular foundation model enabling both efficient distribution modeling and effective generation from only a few context samples. TabGift maps heterogeneous tables into a shared latent space with a pretrained tabular encoder, where an in-context diffusion model learns to denoise latent samples conditioned on a small set of clean examples. This design transfers generative priors across datasets and enables distribution inference from limited context, while a lightweight dataset-specific MLP decoder maps generated representations back to the original data space. Experiments on 7 classification and 11 regression datasets show that TabGift consistently outperforms representative baselines in synthetic data quality, with particularly strong gains under limited-context settings, while achieving substantially faster adaptation and generation.
Yu-Sheng Li, Jun-Peng Jiang, Han-Jia Ye
School of Artificial Intelligence, Nanjing University, China · National Key Laboratory for Novel Software Technology, Nanjing University, China