cs.LGSep 28, 2026

From Data to Program: Fast & Direct Generative Program Inference from Empirical Data

Authors: Simon Klüttermann, Xueying Ding, Leman Akoglu

Organizations: Carnegie Mellon University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Estimating probability densities from a finite set of samples typically requires dataset-specific model fitting. We introduce PRODiGI, a pretrained data-to-program model that infers an explicit, executable generative program in a single forward pass. Pretrained on synthetic datasets paired with their ground-truth programs, PRODiGI accommodates diverse generative families and data dimensionalities through template prediction and non-autoregressive program parameter decoding. Its inferred programs support direct sampling, density and score evaluation, and inspection independently of the pretrained model. We further introduce program-space fine-tuning, which refines differentiable program parameters by matching generated and empirical samples while keeping model parameters intact. Experiments show that PRODiGI achieves lower average density and score MAE than existing pretrained models, while offering multi-fold speedups over its closest competitors. Program-space fine-tuning further reduces generation MMD by 84%. By turning empirical data into explicit, reusable programs, PRODiGI introduces a new direction for fast, interpretable tabular generative modeling.

Figures & tables

Figure 1: PRODiGI overview: (a) Pretraining : Data sampled from normalized, serialized programs across diverse priors is encoded for template prediction and non-autoregressive slot filling; (b) Inference : A single forward pass maps input data to a standalone generative program; (c) Fine-tuning : Inferred program’s differentiable parameters are refined, model parameters remain fixed.
PropertyKDEGMMNDEsTabPFNDiScoDDESNONSNNKDEOurs
Zero-shot/No per-dataset fit✗✗✗✓✓✓✓✓✓✓
Explicit structure and param.s✓✓✗✗✗✗✗✗✓✓
Input-free upon construction✗✓✓✗✗✗✓✓✗✓
Pretrn.-model-free upon constr.✓✓✓✗✗✗✗✗✓✓
Multiple output families✗✗✗✗✗✗✗✗✗✓
Direct generation✓✓△\triangle✓ ∗✗✗✗✓✓✓
Table 1: Comparing classical, neural, and pretrained approaches to density and generative modeling. NDEs: class of Neural Density Estimators; DDE: data-driven Deep Density Estimation ( 27 ) ; SNO: Score Neural Operator ( 21 ) ; NS: Neural Statistician ( 9 ) ; NNKDE: pretrained adaptive KDE ( 37 ) . Last three rows describe capabilities after estimator construction; where direct implies no additional estimation procedure, such as an external sampler, numerical marginalization, etc.
Figure 2: Example input data vs. baseline generations, with corresponding marginals. Appdx. F presents further examples and case studies.
Figure 3: Performance-Inference time trade-off on Generation, Density, and Score estimation tasks on synthetic (all priors) and real-world data. PRODiGI-FT performs best on density and scoring, while PRODiGI offers the fastest generation, remaining competitive on real-world datasets .
 MetricPRODiGI-FTDiscoFormer-FTNNKDE-FTTabPFNKDEGMM
GenerationMMD ( ↓\downarrow ) ×103\times 10^{3}5.46 ± 1.450.89 ± 0.1941.37 ± 8.2067.58 ± 16.5119.51 ± 2.7710.98 ± 1.960.35 ± 0.05 ( ↓\downarrow )5.24 ± 0.950.78 ± 0.30
Rank ( ↓\downarrow )4.29 ± 0.172.92 ± 0.138.05 ± 0.117.93 ± 0.136.73 ± 0.136.14 ± 0.172.39 ± 0.154.33 ± 0.162.23 ± 0.10
% pp << 0.050.05 ( ↓\downarrow )57.3%39.6%99.0%97.9%92.7%80.2%30.2%61.5%31.3%
Time ( ↓\downarrow )0.06 ± 0.0112.90 ± 1.2992.70 ± 0.1493.27 ± 0.140.46 ± 0.261.20 ± 0.0278.09 ± 7.520.79 ± 0.0637.24 ± 2.74
Rank ( ↓\downarrow )1.04 ± 0.024.98 ± 0.037.60 ± 0.058.60 ± 0.052.15 ± 0.063.86 ± 0.057.41 ± 0.143.02 ± 0.066.33 ± 0.05
DensityMAE ( ↓\downarrow )7.58 ± 1.122.19 ± 0.445.10 ± 0.604.17 ± 0.506.44 ± 0.806.25 ± 0.802.31 ± 0.177.14 ± 1.073.46 ± 0.60
Table 2: Performance comparison across tasks and metrics over all prior families a{}^{\text{a}} : (1) Generation distribution match via MMD ( ↓\downarrow ) where %p\%p << 0.050.05 ( ↓\downarrow ) denotes the proportion of datasets where the null hypothesis ( H0H_{0} : generated and real distributions are identical) is rejected; (2) Density estimation via point-wise MAE ( ↓\downarrow ), population-wide Spearman correlation ( ↑\uparrow ), and NLL of generated data ( ↓\downarrow ); (3) Score estimation via MAE ( ↓\downarrow ) and cosine similarity ( ↑\uparrow ). Average rank ( ↓\downarrow ) per metric is across methods over all datasets. Time per task (sec.) is averaged over datasets.
Figure 4: Example real-world data vs. baseline generations, with corresponding marginals for (top) FinancialBeneficiary and (bottom) FinancialTransaction datasets (also see Fig. 11 ).
Figure 5: Ablations of PRODiGI (on 20-dimensional data); left to right: (1) N-AR vs AR decoding validity and speed, (2) generation MMD w/ and w/out normalization, (3) decoding loss w/ and w/out canonical program serialization.
Figure 6: (left) Two-sample rejection rate vs. input dimensionality dd ; (middle) Running time vs. dd ; (right) Average MMD per prior under d≤10d\leq 10 vs. d>10d>10 as the number of context samples is increased.
Appendix figures & tables40 assets

Supplementary material from the paper’s appendix.

Appendix

Method / TaskGenerationDensityScore vector
Neural Statistician [ 9 ]Direct conditional generationNumerical latent marginalizationNumerical marginal-score est.
DDE [ 27 ]External samplerDirect predictionAdditional estimation
Score Neural Operator [ 21 ]Iterative samplingAdditional likelihood computationTime-conditioned score prediction b
Adaptive NNKDE [ 37 ]Direct mixture samplingExplicit kernel densityAnalytic score
TabPFN [ 13 ]Autoregressive samplingAutoregressive product of conditional densitiesNot a native output a
DiScoFormer [ 16 ]External samplerDirect predictionDirect prediction
Appendix
Table 3: Capabilities relevant to generation/samling, density estimation, and score-vector estimation. Additional procedures are distinguished from direct/native model outputs.
HyperparameterValuesDescription
mm[1,10][1,10]Number of mixture components
dd[1,50][1,50]Dimensionality of data
μj(k)\bm{\mu}_{j}^{(k)}∼N(0,5)\sim\mathcal{N}(0,5)Component means
Σjj(k)\mathbf{\Sigma}_{jj}^{(k)}(0.5,2](0.5,2]Diagonal variances
ω\bm{\omega}(0.01,1.0−0.01⋅m⋅(m+1)2](0.01,1.0-0.01\cdot\tfrac{m\cdot(m+1)}{2}]Component weights
Appendix
Table 4: Hyperparameters for Gaussian Mixture Model (GMM) Data Prior
HyperparameterValuesDescription
MD[1,5][1,5]Max. In Degree, Max. number of nodes that
  directly influence any node
MP[1,5][1,5]Max. Path Length, Depth of the graph
dd[1,50][1,50]Dimensionality of data/Number of nodes
wi,jw_{i,j}[0.5,1.5]∪[−1.5,0.5][0.5,1.5]\cup[-1.5,0.5]Edge factors
ηi\eta_{i}[0.1,1.0][0.1,1.0]Gaussian noise factor besides causal relation.
Appendix
Table 5: Hyperparameters for Structural Causal Model (SCM) based Data Priors
Figure 7: SCM activation functions used in this paper. We limit our choice to three functions to make sure their behavior is seperable. Odd represents the Cauchy score functions x1+x2\tfrac{x}{1+x^{2}} and Even represents the Cauchy kernel ( 11+x2\tfrac{1}{1+x^{2}} ).
HyperparameterValuesDescription
CCGaussianCopula parameter family
dd[1,50][1,50]Dimensionality of data/Number of marginals
Marginal{Uniform, Normal, Exponential, \{\text{Uniform, Normal, Exponential, }Marginal family for each feature
 Student t, Kumaraswamy}\text{Student t, Kumaraswamy}\} 
μ\mu / σ\sigma[−5,5][-5,5] / [0.5,2.0][0.5,2.0]Mean/Variance for each marginal
ν\nu[2.5,10][2.5,10]Degrees of freedom for Student’s tt distribution
Appendix
Table 6: Hyperparameters for Copula Data Prior
Figure 8: Examples of each copula marginal used in this paper.
Figure 9: Example inferred programs and generations by PRODiGI for each prior family: (a) A 2-d GMM with 3 components; (b) A 2-d Copula with a uniform and a long-tail marginal, respectively; (c) A 5-d SCM with structural mechanisms odd , even , tanh with (right) ground-truth and (left) inferred parents per node/feature in parentheses (empty implies root variable).
Figure 10: Predicting the family of the program to be reconstructed is a learned task, PRODiGI may occasionally misclassify the data prior. Such confusion is rare, however, with only 1/7441/744 programs misclassified (left). Despite the misclassification, the model continues to fit the data well (right), suggesting that these errors arise from overlapping data priors rather than priors that are inherently difficult for PRODiGI .
Dim=1\text{Dim}=1Dim=2\text{Dim}=2Dim=5\text{Dim}=5
DifferentialExpressionAmusementParkSafetyStudentProgrammePerformance
TrafficIncidentMunicipalExpenditureAbandonedMineLand
HeartMurmurDetectionPlanetaryEmissionsAutoencoderAnomaly
AgilePriorityRxJavaPackageRestaurantViolation
SpeciesValidationFinancialBeneficiaryMaritimeVessel
VotingRecallServiceErrorLocationBusStopPrior
Appendix
Table 7: OddBench datasets used as Real-World Datasets
Figure 11: Example real-world datasets and generations by all methods, including fine-tuned variants. The first six examples are two-dimensional, while the last six are PCA representations of one 5-, 10-, 20-, 30-, 40-, and 50-dimensional dataset each. Real-world manifolds challenge all methods, where PRODiGI-FT notably boosts PRODiGI .
Figure 12: Performance–inference time trade-off for Generation, Density, and Score Estimation on real-world datasets. PRODiGI ranks among the fastest methods across all tasks, while PRODiGI-FT consistently improves performance on each task. However, the limited coverage of prior families constrains performance on datasets that cannot be adequately described by the available priors.
 MetricPRODiGI-FTDiscoFormer-FTNNKDE-FTTabPFNKDEGMM
GenerationMMD ( ↓\downarrow ) ×103\times 10^{3}70.34 ± 5.8636.75 ± 4.56253.32 ± 23.04269.06 ± 22.7680.67 ± 9.2730.95 ± 6.1845.15 ± 10.9469.36 ± 13.2915.59 ± 3.27
Rank ( ↓\downarrow )5.95 ± 0.174.02 ± 0.177.85 ± 0.128.22 ± 0.116.30 ± 0.133.33 ± 0.193.28 ± 0.233.95 ± 0.212.11 ± 0.10
% pp << 0.050.05 ( ↓\downarrow )100%90.2%100%100%100%83.7%84.8%80.4%78.3%
Time ( ↓\downarrow )0.06 ± 0.019.97 ± 1.0393.39 ± 0.1096.05 ± 1.890.10 ± 0.011.16 ± 0.0283.39 ± 7.861.35 ± 0.1117.84 ± 1.14
Rank ( ↓\downarrow )1.13 ± 0.045.07 ± 0.047.61 ± 0.058.62 ± 0.051.89 ± 0.043.60 ± 0.067.52 ± 0.133.41 ± 0.066.15 ± 0.06
DensityMAE ( ↓\downarrow )322 ± 239322 ± 239320 ± 239320 ± 239320 ± 239307 ± 237322 ± 239323 ± 238284 ± 210
Appendix
Table 8: Performance comparison across tasks and metrics on real-world datasets a{}^{\text{a}} : (1) Generation distribution match via MMD ( ↓\downarrow ) where %p\%p << 0.050.05 ( ↓\downarrow ) denotes the proportion of datasets where the null hypothesis ( H0H_{0} : generated and real distributions are identical) is rejected; (2) Density estimation via point-wise MAE ( ↓\downarrow ), population-wide Spearman correlation ( ↑\uparrow ), and NLL of generated data ( ↓\downarrow ); (3) Score estimation via MAE ( ↓\downarrow ) and cosine similarity ( ↑\uparrow ). Average rank ( ↓\downarrow ) per metric is across methods over all datasets. Time ( ↓\downarrow ) per task (s) is averaged over datasets.
Figure 13: Distribution of predicted families on Real-World examples. Each Family is predicted between 14 and 46 times, showing that each of our priors helps to model Real-World data more accurately .
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MMD  (×103)\mathrm{MMD}\;(\times 10^{3})2.81 ± 1.370.38 ± 0.1891.35 ± 42.99276.79 ± 98.848.34 ± 1.250.53 ± 0.280.41 ± 0.130.20 ± 0.130.35 ± 0.18
Rank4.75 ± 0.553.08 ± 0.347.96 ± 0.438.33 ± 0.187.00 ± 0.293.92 ± 0.474.21 ± 0.482.79 ± 0.472.96 ± 0.34
Time0.03 ± 0.001.47 ± 0.1693.28 ± 0.6193.80 ± 0.610.38 ± 0.191.17 ± 0.011.52 ± 0.020.11 ± 0.008.71 ± 0.18
Time Rank1.00 ± 0.004.83 ± 0.238.00 ± 0.009.00 ± 0.003.25 ± 0.244.25 ± 0.175.67 ± 0.142.00 ± 0.007.00 ± 0.00
Dim 2MMD  (×103)\mathrm{MMD}\;(\times 10^{3})1.38 ± 0.900.48 ± 0.2512.26 ± 3.7918.70 ± 8.4766.38 ± 8.588.47 ± 2.720.15 ± 0.070.45 ± 0.330.09 ± 0.08
Rank3.29 ± 0.383.67 ± 0.437.42 ± 0.256.79 ± 0.488.83 ± 0.116.50 ± 0.223.21 ± 0.392.75 ± 0.252.54 ± 0.29
Appendix
Table 9: Distribution match performance via MMD ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families.
DimensionPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 141.7%0.0%91.7%100.0%91.7%25.0%8.3%8.3%8.3%
Dim 216.7%16.7%100.0%83.3%100.0%91.7%16.7%8.3%0.0%
Dim 566.7%41.7%100.0%100.0%91.7%83.3%33.3%75.0%33.3%
Dim 1075.0%50.0%100.0%100.0%100.0%83.3%25.0%66.7%41.7%
Dim 2083.3%58.3%100.0%100.0%91.7%91.7%41.7%83.3%16.7%
Dim 3066.7%58.3%100.0%100.0%100.0%100.0%41.7%91.7%50.0%
Appendix
Table 10: Rejection rate of two-sample tests for distribution match w.r.t. MMD over test datasets from ALL prior families. Tests under generated data that successfully match the input data distribution fail to reject the null (i.e., p>0.05p>0.05 ) that the two samples come from the same distribution.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MMD  (×103)\mathrm{MMD}\;(\times 10^{3})5.31 ± 2.700.45 ± 0.3959.23 ± 41.1376.56 ± 59.006.77 ± 0.951.22 ± 0.710.23 ± 0.110.00 ± 0.000.66 ± 0.45
Rank6.50 ± 0.252.38 ± 0.328.25 ± 0.418.25 ± 0.227.00 ± 0.614.50 ± 0.433.00 ± 0.351.88 ± 0.373.25 ± 0.41
Time0.03 ± 0.001.12 ± 0.0192.61 ± 0.0793.18 ± 0.080.18 ± 0.011.16 ± 0.001.56 ± 0.040.11 ± 0.008.30 ± 0.05
Time Rank1.00 ± 0.004.00 ± 0.008.00 ± 0.009.00 ± 0.003.00 ± 0.005.00 ± 0.006.00 ± 0.002.00 ± 0.007.00 ± 0.00
Dim 2MMD  (×103)\mathrm{MMD}\;(\times 10^{3})4.02 ± 2.150.41 ± 0.3415.23 ± 2.698.24 ± 2.4352.62 ± 17.988.83 ± 2.490.22 ± 0.190.28 ± 0.240.00 ± 0.00
Rank4.62 ± 0.653.38 ± 0.608.00 ± 0.356.75 ± 0.548.75 ± 0.226.25 ± 0.222.75 ± 0.382.50 ± 0.182.00 ± 0.31
Appendix
Table 11: Distribution match performance via MMD ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MMD  (×103)\mathrm{MMD}\;(\times 10^{3})3.07 ± 2.470.36 ± 0.30179.43 ± 108.64538.42 ± 175.539.01 ± 3.470.00 ± 0.000.41 ± 0.220.24 ± 0.170.00 ± 0.00
Rank5.25 ± 0.653.00 ± 0.718.25 ± 0.228.50 ± 0.436.50 ± 0.252.75 ± 0.725.00 ± 0.943.50 ± 0.752.25 ± 0.38
Time0.03 ± 0.001.55 ± 0.0592.65 ± 0.1593.13 ± 0.090.17 ± 0.011.18 ± 0.021.51 ± 0.030.10 ± 0.009.31 ± 0.35
Time Rank1.00 ± 0.005.50 ± 0.258.00 ± 0.009.00 ± 0.003.00 ± 0.004.00 ± 0.005.50 ± 0.252.00 ± 0.007.00 ± 0.00
Dim 2MMD  (×103)\mathrm{MMD}\;(\times 10^{3})0.02 ± 0.020.74 ± 0.647.50 ± 2.7825.72 ± 17.6087.66 ± 4.735.60 ± 3.880.22 ± 0.070.04 ± 0.030.01 ± 0.01
Rank2.62 ± 0.212.88 ± 0.727.25 ± 0.417.25 ± 0.419.00 ± 0.006.25 ± 0.414.50 ± 0.562.88 ± 0.372.38 ± 0.21
Appendix
Table 12: Distribution match performance via MMD ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the SCM prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MMD  (×103)\mathrm{MMD}\;(\times 10^{3})0.06 ± 0.030.34 ± 0.2035.40 ± 12.62215.40 ± 159.879.24 ± 0.440.38 ± 0.190.60 ± 0.270.38 ± 0.330.40 ± 0.19
Rank2.50 ± 0.403.88 ± 0.377.38 ± 1.148.25 ± 0.227.50 ± 0.434.50 ± 0.904.62 ± 0.693.00 ± 0.983.38 ± 0.74
Time0.02 ± 0.001.75 ± 0.4394.58 ± 1.6495.07 ± 1.650.78 ± 0.521.16 ± 0.001.50 ± 0.030.11 ± 0.008.53 ± 0.10
Time Rank1.00 ± 0.005.00 ± 0.358.00 ± 0.009.00 ± 0.003.75 ± 0.653.75 ± 0.225.50 ± 0.252.00 ± 0.007.00 ± 0.00
Dim 2MMD  (×103)\mathrm{MMD}\;(\times 10^{3})0.09 ± 0.050.28 ± 0.1614.03 ± 10.2922.14 ± 16.9758.87 ± 11.9610.98 ± 6.460.00 ± 0.001.02 ± 0.890.25 ± 0.22
Rank2.62 ± 0.414.75 ± 0.547.00 ± 0.356.38 ± 1.248.75 ± 0.227.00 ± 0.352.38 ± 0.542.88 ± 0.623.25 ± 0.65
Appendix
Table 13: Distribution match performance via MMD ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MAE0.34 ± 0.150.10 ± 0.030.49 ± 0.050.43 ± 0.060.21 ± 0.030.19 ± 0.012.38 ± 0.010.15 ± 0.030.12 ± 0.02
Rank4.25 ± 0.841.88 ± 0.367.75 ± 0.237.00 ± 0.004.75 ± 0.154.50 ± 0.409.00 ± 0.003.00 ± 0.712.88 ± 0.28
Time0.29 ± 0.001.48 ± 0.250.03 ± 0.020.64 ± 0.060.17 ± 0.001.15 ± 0.003.59 ± 1.480.14 ± 0.008.42 ± 0.07
Time Rank4.00 ± 0.007.38 ± 0.251.25 ± 0.235.00 ± 0.002.75 ± 0.156.62 ± 0.177.12 ± 0.412.00 ± 0.188.88 ± 0.12
Dim 2MAE0.61 ± 0.160.24 ± 0.080.48 ± 0.070.57 ± 0.070.65 ± 0.050.66 ± 0.021.47 ± 0.190.24 ± 0.020.19 ± 0.04
Rank5.25 ± 1.061.88 ± 0.284.62 ± 0.175.88 ± 0.217.00 ± 0.256.50 ± 0.568.88 ± 0.122.75 ± 0.232.25 ± 0.46
Appendix
Table 14: Density estimation performance via MAE ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MAE0.64 ± 0.230.16 ± 0.050.54 ± 0.080.51 ± 0.070.23 ± 0.050.18 ± 0.012.38 ± 0.010.09 ± 0.010.12 ± 0.01
Rank6.50 ± 0.432.50 ± 0.567.50 ± 0.437.00 ± 0.004.75 ± 0.224.00 ± 0.359.00 ± 0.001.25 ± 0.222.50 ± 0.25
Time0.30 ± 0.001.16 ± 0.020.01 ± 0.000.56 ± 0.000.18 ± 0.011.16 ± 0.005.20 ± 2.610.13 ± 0.008.30 ± 0.05
Time Rank4.00 ± 0.007.00 ± 0.351.00 ± 0.005.00 ± 0.003.00 ± 0.006.50 ± 0.257.75 ± 0.542.00 ± 0.008.75 ± 0.22
Dim 2MAE0.79 ± 0.230.22 ± 0.120.44 ± 0.090.54 ± 0.110.61 ± 0.060.63 ± 0.031.42 ± 0.230.19 ± 0.030.10 ± 0.03
Rank6.50 ± 1.602.25 ± 0.224.50 ± 0.255.75 ± 0.416.50 ± 0.256.25 ± 0.748.75 ± 0.223.00 ± 0.351.50 ± 0.43
Appendix
Table 15: Density estimation performance via MAE ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MAE0.05 ± 0.020.04 ± 0.010.45 ± 0.050.34 ± 0.060.18 ± 0.020.19 ± 0.012.39 ± 0.020.20 ± 0.050.13 ± 0.03
Rank2.00 ± 0.311.25 ± 0.128.00 ± 0.007.00 ± 0.004.75 ± 0.225.00 ± 0.619.00 ± 0.004.75 ± 0.653.25 ± 0.41
Time0.29 ± 0.001.79 ± 0.450.06 ± 0.040.73 ± 0.100.17 ± 0.001.15 ± 0.001.97 ± 0.840.15 ± 0.008.54 ± 0.10
Time Rank4.00 ± 0.007.75 ± 0.221.50 ± 0.435.00 ± 0.002.50 ± 0.256.75 ± 0.226.50 ± 0.432.00 ± 0.359.00 ± 0.00
Dim 2MAE0.42 ± 0.160.27 ± 0.110.52 ± 0.090.59 ± 0.080.70 ± 0.080.68 ± 0.021.53 ± 0.290.29 ± 0.020.29 ± 0.02
Rank4.00 ± 1.061.50 ± 0.434.75 ± 0.226.00 ± 0.007.50 ± 0.256.75 ± 0.829.00 ± 0.002.50 ± 0.253.00 ± 0.61
Appendix
Table 16: Density estimation performance via MAE ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1Corr0.82 ± 0.060.95 ± 0.030.60 ± 0.080.74 ± 0.090.90 ± 0.020.88 ± 0.030.87 ± 0.030.94 ± 0.020.95 ± 0.02
Rank5.21 ± 1.032.79 ± 0.639.00 ± 0.006.43 ± 0.604.57 ± 0.835.43 ± 0.675.86 ± 0.472.71 ± 0.483.00 ± 0.61
Time0.29 ± 0.001.50 ± 0.290.03 ± 0.030.65 ± 0.070.17 ± 0.001.15 ± 0.003.40 ± 1.690.14 ± 0.008.44 ± 0.07
Time Rank4.00 ± 0.007.43 ± 0.281.29 ± 0.265.00 ± 0.002.86 ± 0.136.71 ± 0.177.00 ± 0.451.86 ± 0.138.86 ± 0.13
Dim 2Corr0.76 ± 0.080.91 ± 0.040.80 ± 0.040.80 ± 0.050.92 ± 0.020.79 ± 0.030.73 ± 0.070.95 ± 0.010.93 ± 0.03
Rank5.69 ± 1.112.81 ± 0.617.00 ± 0.436.25 ± 0.584.00 ± 0.316.50 ± 0.617.12 ± 0.822.62 ± 0.433.00 ± 0.71
Appendix
Table 17: Density estimation performance via Spearman correlation between the overall ranking by estimates and ground-truth ( ↑\uparrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1Corr0.68 ± 0.040.91 ± 0.040.50 ± 0.110.64 ± 0.140.92 ± 0.030.90 ± 0.020.87 ± 0.040.96 ± 0.010.95 ± 0.02
Rank7.50 ± 0.253.25 ± 0.969.00 ± 0.007.00 ± 0.613.00 ± 0.795.00 ± 0.615.50 ± 0.562.00 ± 0.352.75 ± 0.54
Time0.30 ± 0.001.16 ± 0.020.01 ± 0.000.56 ± 0.000.18 ± 0.011.16 ± 0.005.20 ± 2.610.13 ± 0.008.30 ± 0.05
Time Rank4.00 ± 0.007.00 ± 0.351.00 ± 0.005.00 ± 0.003.00 ± 0.006.50 ± 0.257.75 ± 0.542.00 ± 0.008.75 ± 0.22
Dim 2Corr0.67 ± 0.140.92 ± 0.060.89 ± 0.050.86 ± 0.070.95 ± 0.020.82 ± 0.040.75 ± 0.070.98 ± 0.000.99 ± 0.00
Rank7.12 ± 1.623.12 ± 0.876.00 ± 0.355.75 ± 0.824.25 ± 0.416.25 ± 0.898.00 ± 0.353.00 ± 0.501.50 ± 0.43
Appendix
Table 18: Density estimation performance via Spearman correlation between the overall ranking by estimates and ground-truth ( ↑\uparrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1Corr1.00 ± 0.001.00 ± 0.000.73 ± 0.050.88 ± 0.060.88 ± 0.030.87 ± 0.050.88 ± 0.020.92 ± 0.040.96 ± 0.03
Rank2.17 ± 0.542.17 ± 0.549.00 ± 0.005.67 ± 0.986.67 ± 0.276.00 ± 1.256.33 ± 0.723.67 ± 0.723.33 ± 1.19
Time0.29 ± 0.001.95 ± 0.570.07 ± 0.050.78 ± 0.120.17 ± 0.001.15 ± 0.001.00 ± 0.040.14 ± 0.008.64 ± 0.05
Time Rank4.00 ± 0.008.00 ± 0.001.67 ± 0.545.00 ± 0.002.67 ± 0.277.00 ± 0.006.00 ± 0.001.67 ± 0.279.00 ± 0.00
Dim 2Corr0.86 ± 0.080.90 ± 0.070.72 ± 0.040.75 ± 0.050.89 ± 0.020.75 ± 0.030.71 ± 0.110.92 ± 0.020.86 ± 0.02
Rank4.25 ± 1.142.50 ± 0.838.00 ± 0.356.75 ± 0.743.75 ± 0.416.75 ± 0.826.25 ± 1.472.25 ± 0.654.50 ± 0.83
Appendix
Table 19: Density estimation performance via Spearman correlation between the overall ranking by estimates and ground-truth ( ↑\uparrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior.
DimensionMetricPRODiGIPRODiGI-FTNNKDENNKDE-FTKDEGMM
Dim 1NLL0.63 ± 0.260.58 ± 0.260.66 ± 0.270.58 ± 0.260.61 ± 0.250.58 ± 0.25
Rank3.81 ± 0.572.69 ± 0.354.75 ± 0.413.62 ± 0.233.25 ± 0.572.88 ± 0.36
Time0.29 ± 0.001.48 ± 0.250.17 ± 0.001.15 ± 0.000.14 ± 0.008.42 ± 0.07
Time Rank3.00 ± 0.004.88 ± 0.121.88 ± 0.124.12 ± 0.121.12 ± 0.126.00 ± 0.00
Dim 2NLL1.46 ± 0.601.32 ± 0.571.55 ± 0.651.50 ± 0.641.31 ± 0.581.29 ± 0.58
Rank3.62 ± 0.572.62 ± 0.374.50 ± 0.374.00 ± 0.223.38 ± 0.212.88 ± 0.30
Appendix
Table 20: Density estimation performance via NLL ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families. We do not report NLL’s for TabPFN and Disco variants, as their predicted densities do not integrate to one.
DimensionMetricPRODiGIPRODiGI-FTNNKDENNKDE-FTKDEGMM
Dim 1NLL0.96 ± 0.450.86 ± 0.460.91 ± 0.490.84 ± 0.460.83 ± 0.450.83 ± 0.46
Rank5.12 ± 0.513.38 ± 0.214.38 ± 0.603.38 ± 0.212.12 ± 0.572.62 ± 0.32
Time0.30 ± 0.001.16 ± 0.020.18 ± 0.011.16 ± 0.000.13 ± 0.008.30 ± 0.05
Time Rank3.00 ± 0.004.75 ± 0.222.00 ± 0.004.25 ± 0.221.00 ± 0.006.00 ± 0.00
Dim 2NLL2.35 ± 0.882.08 ± 0.882.41 ± 0.972.32 ± 0.972.02 ± 0.912.00 ± 0.91
Rank4.12 ± 1.042.38 ± 0.484.88 ± 0.454.12 ± 0.273.12 ± 0.372.38 ± 0.48
Appendix
Table 21: Density estimation performance via NLL ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior. We do not report NLL’s for TabPFN and Disco variants, as their predicted densities do not integrate to one.
DimensionMetricPRODiGIPRODiGI-FTNNKDENNKDE-FTKDEGMM
Dim 1NLL0.30 ± 0.120.30 ± 0.120.41 ± 0.150.33 ± 0.130.39 ± 0.130.32 ± 0.13
Rank2.50 ± 0.402.00 ± 0.475.12 ± 0.513.88 ± 0.374.38 ± 0.603.12 ± 0.62
Time0.29 ± 0.001.79 ± 0.450.17 ± 0.001.15 ± 0.000.15 ± 0.008.54 ± 0.10
Time Rank3.00 ± 0.005.00 ± 0.001.75 ± 0.224.00 ± 0.001.25 ± 0.226.00 ± 0.00
Dim 2NLL0.57 ± 0.490.57 ± 0.490.70 ± 0.610.69 ± 0.600.60 ± 0.520.58 ± 0.50
Rank3.12 ± 0.322.88 ± 0.544.12 ± 0.543.88 ± 0.323.62 ± 0.113.38 ± 0.11
Appendix
Table 22: Density estimation performance via NLL ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior. We do not report NLL’s for TabPFN and Disco variants, as their predicted densities do not integrate to one.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MAE4.66 ± 2.991.00 ± 0.383.15 ± 1.113.33 ± 1.181.86 ± 0.4518.61 ± 4.976.48 ± 1.421.84 ± 0.311.90 ± 0.57
Rank4.69 ± 1.022.44 ± 0.475.25 ± 0.235.88 ± 0.413.38 ± 0.359.00 ± 0.007.25 ± 0.583.62 ± 0.713.50 ± 0.79
Time0.29 ± 0.001.48 ± 0.250.03 ± 0.020.64 ± 0.060.17 ± 0.001.15 ± 0.002.05 ± 0.630.15 ± 0.008.42 ± 0.07
Time Rank4.00 ± 0.007.38 ± 0.251.25 ± 0.235.00 ± 0.002.75 ± 0.156.50 ± 0.187.12 ± 0.332.00 ± 0.189.00 ± 0.00
Dim 2MAE19.34 ± 7.7916.26 ± 7.2028.45 ± 13.8527.94 ± 13.0625.51 ± 12.48102.51 ± 53.9753.68 ± 33.2925.84 ± 12.8728.64 ± 15.15
Rank5.19 ± 0.982.94 ± 0.704.38 ± 0.355.12 ± 0.453.88 ± 0.679.00 ± 0.007.38 ± 0.353.62 ± 0.473.50 ± 0.98
Appendix
Table 23: Score estimation performance via MAE ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MAE9.05 ± 5.111.83 ± 0.464.65 ± 1.944.98 ± 2.032.46 ± 0.7722.28 ± 9.126.89 ± 2.761.82 ± 0.571.69 ± 0.50
Rank7.50 ± 0.253.25 ± 0.745.25 ± 0.226.25 ± 0.223.00 ± 0.509.00 ± 0.006.50 ± 1.031.75 ± 0.412.50 ± 0.56
Time0.30 ± 0.001.16 ± 0.020.01 ± 0.000.56 ± 0.000.18 ± 0.011.16 ± 0.002.73 ± 1.140.14 ± 0.008.30 ± 0.05
Time Rank4.00 ± 0.007.00 ± 0.351.00 ± 0.005.00 ± 0.003.00 ± 0.006.50 ± 0.257.50 ± 0.432.00 ± 0.009.00 ± 0.00
Dim 2MAE5.94 ± 2.952.52 ± 1.293.42 ± 1.683.99 ± 1.913.03 ± 1.4018.47 ± 7.874.72 ± 1.952.54 ± 1.241.51 ± 0.76
Rank6.12 ± 1.352.88 ± 0.914.00 ± 0.355.75 ± 0.544.75 ± 0.749.00 ± 0.007.00 ± 0.614.00 ± 0.791.50 ± 0.43
Appendix
Table 24: Score estimation performance via MAE ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MAE0.26 ± 0.160.17 ± 0.091.64 ± 0.181.68 ± 0.191.26 ± 0.1914.94 ± 2.956.08 ± 0.641.85 ± 0.272.10 ± 1.00
Rank1.88 ± 0.321.62 ± 0.115.25 ± 0.415.50 ± 0.753.75 ± 0.419.00 ± 0.008.00 ± 0.005.50 ± 0.254.50 ± 1.30
Time0.29 ± 0.001.79 ± 0.450.06 ± 0.040.73 ± 0.100.17 ± 0.001.15 ± 0.001.38 ± 0.220.15 ± 0.008.54 ± 0.10
Time Rank4.00 ± 0.007.75 ± 0.221.50 ± 0.435.00 ± 0.002.50 ± 0.256.50 ± 0.256.75 ± 0.412.00 ± 0.359.00 ± 0.00
Dim 2MAE32.74 ± 12.0029.99 ± 10.5453.48 ± 21.2451.88 ± 19.8047.98 ± 19.20186.55 ± 89.77102.64 ± 56.8449.14 ± 19.7455.76 ± 23.44
Rank4.25 ± 1.243.00 ± 1.064.75 ± 0.544.50 ± 0.563.00 ± 0.949.00 ± 0.007.75 ± 0.223.25 ± 0.415.50 ± 1.30
Appendix
Table 25: Score estimation performance via MAE ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1Cosine0.54 ± 0.130.68 ± 0.130.46 ± 0.110.49 ± 0.120.59 ± 0.100.14 ± 0.020.39 ± 0.080.65 ± 0.100.65 ± 0.12
Rank4.94 ± 0.883.12 ± 0.676.25 ± 0.465.50 ± 0.774.38 ± 0.588.31 ± 0.466.12 ± 0.483.38 ± 0.643.00 ± 0.47
Time0.29 ± 0.001.48 ± 0.250.03 ± 0.020.64 ± 0.060.17 ± 0.001.15 ± 0.002.05 ± 0.630.15 ± 0.008.42 ± 0.07
Time Rank4.00 ± 0.007.38 ± 0.251.25 ± 0.235.00 ± 0.002.75 ± 0.156.50 ± 0.187.12 ± 0.332.00 ± 0.189.00 ± 0.00
Dim 2Cosine0.59 ± 0.110.80 ± 0.080.66 ± 0.090.63 ± 0.090.75 ± 0.060.22 ± 0.020.54 ± 0.050.76 ± 0.060.77 ± 0.08
Rank5.69 ± 1.103.31 ± 0.825.38 ± 0.396.00 ± 0.533.75 ± 0.668.75 ± 0.155.88 ± 0.623.50 ± 0.402.75 ± 0.81
Appendix
Table 26: Score estimation performance via cosine similarity between score vector estimates and ground-truth ( ↑\uparrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1Cosine0.35 ± 0.080.62 ± 0.150.45 ± 0.100.45 ± 0.100.70 ± 0.100.15 ± 0.020.51 ± 0.090.76 ± 0.060.73 ± 0.08
Rank6.88 ± 0.873.25 ± 1.146.25 ± 0.416.75 ± 0.653.00 ± 0.008.88 ± 0.115.50 ± 0.562.00 ± 0.612.50 ± 0.43
Time0.30 ± 0.001.16 ± 0.020.01 ± 0.000.56 ± 0.000.18 ± 0.011.16 ± 0.002.73 ± 1.140.14 ± 0.008.30 ± 0.05
Time Rank4.00 ± 0.007.00 ± 0.351.00 ± 0.005.00 ± 0.003.00 ± 0.006.50 ± 0.257.50 ± 0.432.00 ± 0.009.00 ± 0.00
Dim 2Cosine0.50 ± 0.160.87 ± 0.090.86 ± 0.050.81 ± 0.070.87 ± 0.020.26 ± 0.020.61 ± 0.070.90 ± 0.010.97 ± 0.01
Rank6.88 ± 1.563.38 ± 1.084.50 ± 0.255.25 ± 0.654.50 ± 1.038.50 ± 0.256.50 ± 0.754.00 ± 0.611.50 ± 0.43
Appendix
Table 27: Score estimation performance via cosine similarity between score vector estimates and ground-truth ( ↑\uparrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1Cosine0.73 ± 0.210.73 ± 0.210.47 ± 0.200.54 ± 0.210.48 ± 0.150.14 ± 0.040.27 ± 0.080.53 ± 0.170.57 ± 0.21
Rank3.00 ± 0.683.00 ± 0.686.25 ± 0.824.25 ± 1.085.75 ± 0.657.75 ± 0.826.75 ± 0.654.75 ± 0.543.50 ± 0.75
Time0.29 ± 0.001.79 ± 0.450.06 ± 0.040.73 ± 0.100.17 ± 0.001.15 ± 0.001.38 ± 0.220.15 ± 0.008.54 ± 0.10
Time Rank4.00 ± 0.007.75 ± 0.221.50 ± 0.435.00 ± 0.002.50 ± 0.256.50 ± 0.256.75 ± 0.412.00 ± 0.359.00 ± 0.00
Dim 2Cosine0.69 ± 0.130.73 ± 0.120.46 ± 0.100.45 ± 0.110.62 ± 0.070.18 ± 0.030.47 ± 0.060.62 ± 0.070.57 ± 0.09
Rank4.50 ± 1.303.25 ± 1.246.25 ± 0.416.75 ± 0.653.00 ± 0.619.00 ± 0.005.25 ± 0.893.00 ± 0.354.00 ± 1.27
Appendix
Table 28: Score estimation performance via cosine similarity between score vector estimates and ground-truth ( ↑\uparrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MMD  (×103)\mathrm{MMD}\;(\times 10^{3})41.79 ± 15.476.39 ± 2.24184.48 ± 66.92213.03 ± 81.6619.72 ± 3.366.30 ± 3.407.59 ± 3.515.03 ± 2.525.93 ± 3.09
Rank5.90 ± 0.623.85 ± 0.688.60 ± 0.158.30 ± 0.205.80 ± 0.283.10 ± 0.543.70 ± 0.552.70 ± 0.213.05 ± 0.34
Time0.04 ± 0.011.22 ± 0.0693.20 ± 0.1493.87 ± 0.150.08 ± 0.001.13 ± 0.002.12 ± 0.230.22 ± 0.028.34 ± 0.10
Time Rank1.10 ± 0.094.70 ± 0.148.00 ± 0.009.00 ± 0.002.00 ± 0.144.30 ± 0.146.00 ± 0.002.90 ± 0.097.00 ± 0.00
Dim 2MMD  (×103)\mathrm{MMD}\;(\times 10^{3})65.60 ± 12.6818.19 ± 8.0474.69 ± 16.1288.65 ± 23.06165.03 ± 39.658.53 ± 2.754.86 ± 1.488.73 ± 5.712.94 ± 0.84
Rank6.42 ± 0.634.42 ± 0.536.92 ± 0.307.17 ± 0.428.25 ± 0.293.46 ± 0.513.04 ± 0.302.38 ± 0.502.96 ± 0.34
Appendix
Table 29: Distribution match performance via MMD ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets.
DimensionPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1100.0%60.0%100.0%100.0%100.0%50.0%60.0%50.0%40.0%
Dim 2100.0%66.7%100.0%100.0%100.0%58.3%66.7%41.7%50.0%
Dim 5100.0%90.9%100.0%100.0%100.0%81.8%72.7%63.6%63.6%
Dim 10100.0%100.0%100.0%100.0%100.0%72.7%90.9%81.8%72.7%
Dim 20100.0%100.0%100.0%100.0%100.0%100.0%83.3%100.0%100.0%
Dim 30100.0%100.0%100.0%100.0%100.0%100.0%100.0%100.0%91.7%
Appendix
Table 30: Rejection rate of two-sample tests for distribution match w.r.t. MMD over test datasets over real-world test datasets.. Tests under generated data that successfully match the input data distribution fail to reject the null (i.e., p>0.05p>0.05 ) that the two samples come from the same distribution.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MAE1035 ± 7911035 ± 7911037 ± 7911037 ± 7911037 ± 791991 ± 7891037 ± 7901045 ± 789933 ± 695
Rank6.10 ± 0.684.30 ± 0.755.70 ± 0.606.30 ± 0.455.80 ± 0.633.70 ± 0.726.20 ± 0.884.50 ± 0.932.40 ± 0.62
Time0.04 ± 0.011.22 ± 0.060.02 ± 0.010.68 ± 0.060.08 ± 0.001.13 ± 0.001.07 ± 0.020.27 ± 0.028.34 ± 0.10
Time Rank2.00 ± 0.147.40 ± 0.321.30 ± 0.285.30 ± 0.282.80 ± 0.137.00 ± 0.206.30 ± 0.143.90 ± 0.099.00 ± 0.00
Dim 2MAE5 ± 13 ± 13 ± 13 ± 12 ± 13 ± 13 ± 13 ± 12 ± 1
Rank6.78 ± 0.865.00 ± 0.834.67 ± 0.475.33 ± 0.425.56 ± 0.694.56 ± 0.827.22 ± 0.684.44 ± 0.861.44 ± 0.28
Appendix
Table 31: Density estimation performance via MAE ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1Corr0.24 ± 0.090.41 ± 0.100.31 ± 0.090.37 ± 0.110.51 ± 0.110.74 ± 0.090.43 ± 0.110.60 ± 0.130.49 ± 0.12
Rank6.65 ± 0.675.20 ± 0.646.75 ± 0.546.15 ± 0.674.60 ± 0.382.80 ± 0.565.30 ± 0.723.05 ± 0.704.50 ± 0.74
Time0.04 ± 0.011.22 ± 0.060.02 ± 0.010.68 ± 0.060.08 ± 0.001.13 ± 0.001.07 ± 0.020.27 ± 0.028.34 ± 0.10
Time Rank2.00 ± 0.147.40 ± 0.321.30 ± 0.285.30 ± 0.282.80 ± 0.137.00 ± 0.206.30 ± 0.143.90 ± 0.099.00 ± 0.00
Dim 2Corr0.40 ± 0.080.59 ± 0.080.54 ± 0.060.53 ± 0.060.54 ± 0.060.57 ± 0.060.54 ± 0.090.64 ± 0.050.76 ± 0.07
Rank7.50 ± 0.435.22 ± 0.735.67 ± 0.655.44 ± 0.744.78 ± 0.805.67 ± 0.655.72 ± 0.933.33 ± 0.741.67 ± 0.27
Appendix
Table 32: Density estimation performance via Spearman correlation between the overall ranking by estimates and ground-truth ( ↑\uparrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets.
DimensionMetricPRODiGIPRODiGI-FTNNKDENNKDE-FTKDEGMM
Dim 1NLL1.26 ± 0.121.29 ± 0.201.14 ± 0.0949.30 ± 28.9811.37 ± 8.79105.05 ± 96.95
Rank3.50 ± 0.552.40 ± 0.403.80 ± 0.564.40 ± 0.432.80 ± 0.564.10 ± 0.41
Time0.04 ± 0.011.22 ± 0.060.08 ± 0.001.13 ± 0.000.27 ± 0.028.34 ± 0.10
Time Rank1.10 ± 0.094.70 ± 0.141.90 ± 0.094.30 ± 0.143.00 ± 0.006.00 ± 0.00
Dim 2NLL4.02 ± 0.812.38 ± 0.491.90 ± 0.361.53 ± 0.391.51 ± 0.341.48 ± 0.39
Rank5.00 ± 0.474.11 ± 0.533.78 ± 0.472.89 ± 0.372.67 ± 0.352.56 ± 0.45
Appendix
Table 33: Density estimation performance via NLL ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets. We do not report NLL’s for TabPFN and Disco variants, as their predicted densities do not integrate to one.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1MAE14362 ± 987214361 ± 987114361 ± 987214361 ± 987214361 ± 987213251 ± 974014358 ± 986914344 ± 985911859 ± 7643
Rank6.30 ± 0.454.90 ± 0.505.70 ± 0.695.70 ± 0.474.40 ± 0.867.00 ± 0.994.70 ± 0.573.70 ± 0.852.60 ± 0.79
Time0.04 ± 0.011.22 ± 0.060.02 ± 0.010.68 ± 0.060.08 ± 0.001.13 ± 0.001.27 ± 0.060.31 ± 0.028.35 ± 0.10
Time Rank2.00 ± 0.147.00 ± 0.321.30 ± 0.285.20 ± 0.192.80 ± 0.136.40 ± 0.257.40 ± 0.153.90 ± 0.099.00 ± 0.00
Dim 2MAE162 ± 98170 ± 98511 ± 291320 ± 140240 ± 1055297 ± 3254164 ± 99162 ± 9946 ± 16
Rank6.11 ± 0.714.78 ± 0.835.56 ± 0.525.33 ± 0.524.22 ± 0.498.22 ± 0.734.78 ± 0.913.89 ± 0.672.11 ± 0.62
Appendix
Table 34: Score estimation performance via MAE ( ↓\downarrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets.
DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM
Dim 1Cosine0.04 ± 0.030.17 ± 0.070.13 ± 0.050.04 ± 0.020.24 ± 0.070.37 ± 0.120.29 ± 0.100.49 ± 0.100.42 ± 0.11
Rank7.05 ± 0.506.05 ± 0.466.35 ± 0.367.35 ± 0.324.70 ± 0.654.05 ± 0.854.90 ± 0.672.00 ± 0.602.55 ± 0.36
Time0.04 ± 0.011.22 ± 0.060.02 ± 0.010.68 ± 0.060.08 ± 0.001.13 ± 0.001.27 ± 0.060.31 ± 0.028.35 ± 0.10
Time Rank2.00 ± 0.147.00 ± 0.321.30 ± 0.285.20 ± 0.192.80 ± 0.136.40 ± 0.257.40 ± 0.153.90 ± 0.099.00 ± 0.00
Dim 2Cosine0.10 ± 0.040.28 ± 0.050.26 ± 0.080.25 ± 0.070.27 ± 0.070.37 ± 0.080.23 ± 0.060.35 ± 0.070.61 ± 0.10
Rank7.28 ± 0.654.67 ± 0.725.33 ± 0.725.50 ± 0.665.06 ± 0.864.67 ± 0.966.72 ± 0.643.67 ± 0.542.11 ± 0.55
Appendix
Table 35: Score estimation performance via cosine similarity between score vector estimates and ground-truth ( ↑\uparrow ) and rank, as well as runtime in seconds ( ↓\downarrow ) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets.

Explore similar work

CardsList

  1. Probabilistic Programs of Thought

    Apr 19, 2026Poorva Garg, Renato Lui Geh, Daniel Israel +3Code GenerationLLM Reasoning Strategies

    University of California Los Angeles · Allen Institute for AI

  2. Synthesis Without Training: An Inference-Only Pipeline for Tabular, Temporal, and Relational Synthetic Data

    Sep 29, 2026Zilong Zhao, Abdul Raheem, Jiayu Li +5Synthetic Tabular DataSynthetic Data

    Betterdata AI · National University of Singapore · University of Illinois Urbana-Champaign +1

  3. Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation

    Jul 29, 2026Alexi Gladstone, Heng Ji, Yilun DuGenerative ModelsPretraining

    1UIUC · 2Harvard