DiffLTM: Continuous Diffusions Large Tabular Models for Conditional Generation

Alankrita Bhatt; Mukur Gupta; Andrea Montanari; Pulkit Tandon

cs.LGOct 21, 2026

DiffLTM: Continuous Diffusions Large Tabular Models for Conditional Generation

Authors: Alankrita Bhatt, Mukur Gupta, Andrea Montanari, Pulkit Tandon

Organizations: Granica Computing Inc. · Stanford University

Abstract

Conditional generation is a key capability to be supported by Large Tabular Models, enabling a number of applications. Accurate conditional generation has to contend with two key challenges: $(i)$ Data comprise a mixture of numerical and categorical entries, the latter often with high cardinality and heavily unbalanced proportions; $(ii)$ Conditional generation becomes increasingly difficult for rare sub-populations. The best models in this class use a hybrid of continuous and discrete diffusions. In this paper, we design a continuous diffusion model with strong generation quality. An important element is the use of cross-entropy loss for categoricals, for which we provide a first theoretical justification. We introduce metrics that focus on rare subpopulations. Further, we generalize conditional generation techniques to enable new types of conditioning. Finally, we show that our model (trained on labor data) can capture classical insights from econometrics. Code is available at https://github.com/granica-ai/DiffLTM

Figures & tables

Figure 1: Conditional Shape score for conditional generation of ACS Labour data (

n_{\mbox{\tiny\rm train}}\approx 4.8

M). For each condition

\mathcal{C}

, scores are evaluated on matched synthetic and test samples of size

n_{\mbox{\tiny\rm synth}}=n_{\mbox{\tiny\rm test}}=\min(n_{\mbox{\tiny\rm test}}^{\mathcal{C}},100{,}000)

where

n_{\mbox{\tiny\rm test}}^{\mathcal{C}}

denotes the number of rows in the test set that satisfy

\mathcal{C}

. DiffLTM -CFG achieves accuracy very close to Rejection sampling (ideal conditioning using samples from DiffLTM ) and to Ideal , outperforming DiffLTM -SMC and Harpoon (Section 4 ; Appendix C.3 ).

Figure 2: Evolution of 10th percentile hourly wage in Arizona and Pennsylvania: observed wages display the effect of the Arizona minimum wage increase post-2016. Comparison of counterfactuals based on synthetic controls and conditional generation. Trained without each state’s post-2016 rows (shaded), DiffLTM -CFG predicts the held-out Pennsylvania wages (placebo, right), and in Arizona (left) produces a no-policy counterfactual well below the observed post-2017 wages. (Section 5 ; Appendix D ).

Dataset	Architecture	Shape	Trend	CShape	MLE
ACSLabour	MSE+Repr	89.39 (0.18)	79.61 (0.31)	72.36 (0.18)	91.22 (1.21)
	MSE+Logits	93.70 (0.48)	87.40 (0.80)	81.40 (0.91)	97.33 (0.43)
	CE+Logits	98.72 (0.06)	96.87 (0.10)	94.87 (0.07)	99.10 (0.02)
	CE+Logits+Learnt	99.04 (0.04)	97.66 (0.04)	95.37 (0.04)	99.41 (0.05)
ACSPubCov	MSE+Repr	98.61 (0.35)	97.32 (0.63)	95.56 (1.73)	98.65 (0.69)
	MSE+Logits	98.84 (0.10)	97.80 (0.15)	97.01 (1.06)	99.17 (0.28)

Table 2: Comparison across architectural variants, including MSE versus CE loss, representation (Repr) versus logits parameterization, and the effect of learned embeddings. Higher values indicate better performance for all metrics (mean (SD) over three training seeds) If multiple approaches are within

0.3\%

of the best mean, all are underlined and none is bold; otherwise the best is bold and the second-best underlined.

Figure 3: The counterfactual effect of Proposition 206 tracks each occupation’s exposure to the minimum wage. The gap between dashed curves (i.e. true 2019 values and DiffLTM predicted counterfactual) is large for cleaning and sales workers, whose observed wages pile up at $11, and negligible for healthcare workers, whose wages already sat above it (see Appendix D.4 for details).

Figure 4: Counterfactual analysis using synthetic control and conditional generation (based on DiffLTM ) for data generated according to a structural equations model. The prediction for

f(y)=y

\log y

y(\max y-y)

Figure 5: Squared error between Mincer coefficients fitted on conditionally generated sub-populations and real data slices. Here we used

n^{\mbox{\tiny\rm synth}}=n^{\mbox{\tiny\rm test}}\approx\min\left(n_{\mbox{\tiny\rm test}}^{\mathcal{C}},100{,}000\right)

, where

n_{\mbox{\tiny\rm test}}^{\mathcal{C}}

denotes the number of eligible test rows satisfying

\mathcal{C}

Figure 6: Returns to education (China-born, Female, Private Industry). For China-born female private-sector workers, both methods reproduce the education ordering and the concave age–earnings profile of the real data.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Column	Type	Description
AGEP	Numerical	Age in completed years.
WKHP	Numerical	Usual hours worked per week during the previous 12 months.
log_wage	Numerical	Regression target: natural logarithm of annual wage and salary income in constant 2018 dollars, computed as $\log(\texttt{WAGP}\times\texttt{ADJINC}/10^{6})$ .
ST	Categorical	State or territory of residence.
PUMA	Categorical	Public Use Microdata Area of residence; codes are unique only within state.
SEX	Categorical	Sex.

Appendix

Table 3: Modeled columns in ACSLabour. Categorical variables retain their original ACS PUMS codes; code definitions and semantic labels are provided in the ACS 2014–2018 PUMS Data Dictionary [ 37 ] .

Dataset	Rows	Numerical	Categorical	Target	Task
ACSLabour	$6{,}892{,}317$	$3$	$15$	log_wage	Regression
ACSPubCov [ 13 ]	$5{,}916{,}565$	$2$	$18$	PUBCOV	Classification
ACSMobility [ 13 ]	$2{,}041{,}325$	$4$	$17$	MIG	Classification

Appendix

Table 4: Summary of the ACS-derived datasets

Figure 7: The DiffLTM conditional denoiser differs from the unconditional one only at the input. For each column, the noisy state is concatenated with the parameters of every condition involving that column (equality value, range bounds, or linear weight), each paired with a mask indicating whether the condition is present. A linear condition additionally contributes its bounds and its value

s_{t}

at the current noisy state. The concatenated input is then projected and processed by the same time-conditioned MLP and output heads as the unconditional model, and setting all masks to zero recovers the unconditional model (Section 4.1 ; Appendix C.1 ).

Figure 8: Shape score for conditional generation of ACS Labour data (

n_{\mbox{\tiny\rm train}}\approx 4.8

M) with Harpoon implemented on same base model as CFG and SMC. The generation and evaluation protocol is otherwise identical to that followed in Figure 1 .

Figure 9: DiffLTM generates the full counterfactual wage distribution for any subgroup. Arizona wage CDFs (lowest 30%); 2016 seen in training, 2019 held out. Dotted: $11 minimum set by Proposition 206. The policy effect, the gap between dashed curves, is large for high-school workers and small for bachelor’s-degree workers relative to their 2016 gap (Section 5 ; Appendix D.4 ).

Fidelity Metric	Dataset	Figure
Shape	ACSLabour	Figure 10
Trend	ACSLabour	Figure 11
Conditional Shape (Rare)	ACSLabour	Figure 12
Shape	ACSPubCov	Figure 13
Trend	ACSPubCov	Figure 14
Conditional Shape	ACSPubCov	Figure 15

Appendix

Table 5: Additional fidelity vs. rarity results across fidelity metrics and datasets.

Figure 10: Shape vs rarity on ACSLabour Dataset.

Figure 11: Trend vs rarity on ACSLabour Dataset.

Figure 12: Conditional Shape (Rare) vs rarity on ACSLabour Dataset.

Figure 13: Shape vs rarity on ACSPubCov Dataset.

Figure 14: Trend vs rarity on ACSPubCov Dataset.

Figure 15: Conditional Shape vs rarity on ACSPubCov Dataset.

Figure 16: Conditional Shape (Rare) vs rarity on ACSPubCov Dataset.

Figure 17: Shape vs rarity on ACSMobility Dataset.

Figure 18: Trend vs rarity on ACSMobility Dataset.

Figure 19: Conditional Shape vs rarity on ACSMobility Dataset.

Figure 20: Conditional Shape (Rare) vs rarity on ACSMobility Dataset.

Slice	Rarity (%)	Shape $\times 100$ (SMC)	Shape $\times 100$ (Ideal)
$\left(\frac{\mathrm{AGEP}-43}{10}\right)^{2}+\left(\frac{\texttt{log\_wage}-10.6}{0.5}\right)^{2}\leq 0.64$	9.48	97.56	99.09
$0.25\leq\left(\frac{\mathrm{AGEP}-43}{10}\right)^{2}+\left(\frac{\texttt{log\_wage}-10.6}{0.5}\right)^{2}\leq 0.64$	5.91	97.32	98.97
$(\mathrm{AGEP}-43)(\mathrm{WKHP}-40)<-200$	6.90	96.75	99.08
$60\leq\frac{\exp(\texttt{log\_wage})}{52\,\mathrm{WKHP}}\leq 90$	3.63	96.70	98.73

Appendix

Table 6: Conditional generation performance of SMC for arbitrary (nonlinear) numerical conditions on the ACSLabour dataset. Shape values displayed are means over three training seeds. Details on implementation and generation provided in Appendix C.2 .

Tabular Diffusion Models

Tabular diffusion models generate or impute mixed numerical and categorical tables by reversing learned noise processes.

See all 25 papers

TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion
Feb 26, 2026Donghong Cai, Jiarui Feng, Yanbo Wang +3Synthetic Tabular Data Generation Tabular Diffusion Models
Washington University in St. Louis · Peking University · Ant Group
CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data
Sep 30, 2026Mohamed Amine Ketata, Maximilian Schambach, Stephan GünnemannMasked Diffusion Models Tabular Diffusion Models
Munich Data Science Institute, Technical University of Munich, Germany · SAP SE, Germany
TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks
Oct 11, 2026Jayoung Kim, Sanghyun Hong, Noseong ParkSynthetic Tabular Data Generation Tabular Diffusion Models
KAIST · Oregon State University

Feb 26, 2026J/K move · Enter open · S save

TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion

Donghong Cai, Jiarui Feng, Yanbo Wang, Da Zheng+2

Washington University in St. Louis · Peking University · Ant Group

Synthetic Tabular Data Generation Tabular Diffusion Models Multimodal Generation Continuous Diffusion Language Models

Open paper AskPDF arXiv

Synthetic tabular data generation has attracted growing attention due to its importance for data augmentation, foundation models, and privacy. However, real-world tabular datasets increasingly contain free-form text fields (e.g., reviews or clinical notes) alongside structured numerical and categorical attributes. Generating such heterogeneous tables with joint modeling of different modalities remains challenging. Existing approaches broadly fall into two categories: diffusion-based methods and LLM-based methods. Diffusion models can capture complex dependencies over numerical and categorical features in continuous or discrete spaces, but extending them to open-ended text is nontrivial and often leads to degraded text quality. In contrast, LLM-based generators naturally produce fluent text, yet their discrete tokenization can distort precise or wide-range numerical values, hindering accurate modeling of both numbers and language. In this work, we propose TabDLM, a unified framework for free-form tabular data generation via a joint numerical-language diffusion model built on masked diffusion language models (MDLMs). TabDLM models textual and categorical features through masked diffusion, while modeling numerical features with a continuous diffusion process through learned specialized numeric tokens embedding; bidirectional attention then captures cross-modality interactions within a single model. Extensive experiments on diverse benchmarks demonstrate the effectiveness of TabDLM compared to strong diffusion- and LLM-based baselines.

Abstract

Tabular Diffusion Models

Explore similar work

TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion

CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data

TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks