TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion

Donghong Cai; Jiarui Feng; Yanbo Wang; Da Zheng; Yixin Chen; Muhan Zhang

cs.LGFeb 26, 2026

TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion

Authors: Donghong Cai, Jiarui Feng, Yanbo Wang, Da Zheng, Yixin Chen, Muhan Zhang

Organizations: Washington University in St. Louis · Peking University · Ant Group

Abstract

Synthetic tabular data generation has attracted growing attention due to its importance for data augmentation, foundation models, and privacy. However, real-world tabular datasets increasingly contain free-form text fields (e.g., reviews or clinical notes) alongside structured numerical and categorical attributes. Generating such heterogeneous tables with joint modeling of different modalities remains challenging. Existing approaches broadly fall into two categories: diffusion-based methods and LLM-based methods. Diffusion models can capture complex dependencies over numerical and categorical features in continuous or discrete spaces, but extending them to open-ended text is nontrivial and often leads to degraded text quality. In contrast, LLM-based generators naturally produce fluent text, yet their discrete tokenization can distort precise or wide-range numerical values, hindering accurate modeling of both numbers and language. In this work, we propose TabDLM, a unified framework for free-form tabular data generation via a joint numerical-language diffusion model built on masked diffusion language models (MDLMs). TabDLM models textual and categorical features through masked diffusion, while modeling numerical features with a continuous diffusion process through learned specialized numeric tokens embedding; bidirectional attention then captures cross-modality interactions within a single model. Extensive experiments on diverse benchmarks demonstrate the effectiveness of TabDLM compared to strong diffusion- and LLM-based baselines.

Figures & tables

Figure 1: Overview of TabDLM . During training, given an input tabular sample, TabDLM applies masked language diffusion to categorical and free-form textual features, and continuous diffusion to numerical features. Noisy inputs are mapped to token embeddings via a textual embedding layer and a numerical encoder. An MDLM then denoises these embeddings to reconstruct the original sample. Training updates only the projector in the numerical encoder and decoder, as well as the LoRA modules within each MDLM layer.

Method	MathExpr				ProfileBio
Method	Shape $\downarrow$	Trend $\downarrow$	Op-MR $\uparrow$	Exp-MR $\uparrow$	Shape $\downarrow$	Trend $\downarrow$	Bio-MR $\uparrow$
Qwen2.5-7B ${}_{\text{ICL}}$	$39.04$	$56.65$	$53.93$	$53.92$	$41.29$	$64.52$	$7.97$
Qwen2.5-14B ${}_{\text{ICL}}$	$25.61$	$36.07$	$76.01$	$75.12$	$43.97$	$65.00$	$13.00$
Qwen2.5-7B ${}_{\text{SFT}}$	$5.07$	$50.81$	$99.68$	$99.34$	$10.51$	$36.26$	$98.51$
LLaDA-8B ${}_{\text{SFT}}$	$5.41$	$42.54$	$98.36$	$97.90$	$5.81$	$44.14$	$97.63$
TabDLM	3.69	6.07	99.99	99.80	4.84	7.81	98.60

Table 1: Evaluation results on the MathExpr and ProfileBio datasets (%).

Method	Amazon				Arxiv
Method	Shape (%) $\downarrow$	Trend (%) $\downarrow$	$\text{MLE}^{*}$ $\uparrow$	MLE $\uparrow$	Shape (%) $\downarrow$	Trend (%) $\downarrow$	$\text{MLE}^{*}$ $\uparrow$	MLE $\uparrow$
Real	$0.0$	$0.0$	$.905$	$.906$	$0.0$	$0.0$	$.968$	$.968$
Qwen2.5-7B ${}_{\text{ICL}}$	$37.93$	$68.27$	$.636$	$.629$	$68.04$	$42.34$	$.521$	$.516$
Qwen2.5-14B ${}_{\text{ICL}}$	$39.19$	$78.22$	$.684$	$.679$	$40.79$	$34.35$	$.556$	$.527$
Qwen2.5-7B ${}_{\text{SFT}}$	$11.41$	$46.31$	$.824$	$.834$	$11.19$	$10.02$	$.701$	$.727$
LLaDA-8B ${}_{\text{SFT}}$	$7.76$	$37.13$	$.891$	$.892$	$14.10$	$43.81$	$.931$	$.930$

Table 2: Results on the Amazon and Arxiv datasets. ( ∗ indicates (w/o nums) )

Method	Adult	Default	Shoppers	Magic	Beijing	Average
CTGAN	16.84 / 20.23	16.83 / 26.95	21.15 / 13.08	9.81 / 7.00	21.39 / 22.95	17.20 / 18.04
TVAE	14.22 / 14.15	10.17 / 19.50	24.51 / 18.67	8.25 / 5.82	19.16 / 18.01	15.26 / 15.23
GOGGLE	16.97 / 45.29	17.02 / 21.94	22.33 / 23.90	1.90 / 9.47	16.93 / 45.94	15.03 / 29.31
STaSy	11.29 / 14.51	5.77 / 5.96	9.37 / 8.49	6.29 / 6.61	6.71 / 8.00	7.89 / 8.71
CoDi	21.38 / 22.49	15.77 / 68.41	31.84 / 17.78	11.56 / 6.53	16.94 / 7.07	19.50 / 24.46
TabDDPM	1.75 / 3.01	1.57 / 4.89	2.72 / 6.61	1.01 / 1.70	1.30 / 2.71	1.67 / 3.78

Table 3: Performance comparison on the error rates (%) of Shape

\downarrow

/ Trend

\downarrow

. Each cell reports Shape / Trend. Full per-metric results are provided in Appendix C.1 . ( B : best overall; B : best in language-based model)

Method	Shape $\downarrow$	Trend $\downarrow$	MLE $\uparrow$	C2ST $\uparrow$	$\alpha$ -Precision $\uparrow$
TabDLM - noFloatAE	$2.15$	$4.34$	$.870$	$0.9256$	$98.50$
TabDLM - onlyContDiff	$4.19$	$7.28$	$.850$	$0.8399$	$91.70$
TabDLM - onlyMDLM	$5.75$	$22.96$	$.852$	$0.7892$	$88.58$
TabDLM	$\mathbf{1.56}$	$\mathbf{2.49}$	$\mathbf{.874}$	$\mathbf{.9571}$	$\mathbf{98.56}$

Table 4: Ablation study on the average performance across Adult, Default, and Shoppers

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Column	Example Value
$x_{1}$	$2.75$
$x_{2}$	$6.40$
operation_x1	sin
operation_x2	log
operation_between	mul
latex_expression	\sin(2.75) \times \log(6.40)

Appendix

Table 5: An example sample from the MathExpr dataset.

Column	Example Value
age	$38$
salary	$135$
sex	female
birth_state	California
college	Harvard University
degree	master

Appendix

Table 6: An example sample from the ProfileBio dataset.

Component	Rule / Template
Age descriptor	[21, 30]: in the early career stage [31, 40]: in the career-building stage [41, 50]: in the established career stage [51, 60]: in the advanced career stage [61, 70]: in the late career stage
Salary descriptor	[75, 100]: a comfortable income [101, 150]: a strong professional income [151, 200]: a high-level income
Biography template	This {sex} individual is {age_desc}. {He/She} was born in {birth_state} and completed higher education at {college}, earning a {degree} degree. {He/She} works as a {occupation}. {He/She} earns {salary_desc}.

Appendix

Table 7: ProfileBio: Age/salary mapping rules and the biography template.

Column	Example Value
price	$5.98$
review_time	$1970$ (days since earliest review date)
rating	$5.0$
verified	true
category	Science Fiction & Fantasy > Fantasy
brand	Visit Amazon’s J. R. R. Tolkien Page

Appendix

Table 8: An example sample from the Amazon dataset.

Column	Example Value
submission_time	$1672$ (days since earliest review date)
submission_year	$2022$
title_length	$7$
abstract_length	$90$
category	Category_3
title	IRC-safe Graph Autoencoder for unsupervised anomaly detection

Appendix

Table 9: An example record from the Arxiv dataset.

Dataset	#Rows	#Num	#Cat	#Max Cat	#Train	#Validation	#Test	Task
Adult	48,842	6	9	42	28,943	3,618	16,281	Classification
Default	30,000	14	11	11	24,000	3,000	3,000	Classification
Shoppers	12,330	10	8	20	9,864	1,233	1,233	Classification
Magic	19,019	10	1	2	15,215	1,902	1,902	Classification
Beijing	43,824	7	5	31	35,058	4,383	4,383	Regression

Appendix

Table 10: Statistics of real-world tabular datasets. #Num denotes the number of numerical columns, and #Cat denotes the number of categorical columns. #Max Cat is the maximum number of categories among all categorical columns.

Method	Adult	Default	Shoppers	Magic	Beijing	Average
CTGAN	$16.84\pm 0.03$	$16.83\pm 0.04$	$21.15\pm 0.10$	$9.81\pm 0.08$	$21.39\pm 0.05$	$17.20$
TVAE	$14.22\pm 0.08$	$10.17\pm 0.05$	$24.51\pm 0.06$	$8.25\pm 0.06$	$19.16\pm 0.06$	$15.26$
GOGGLE	$16.97$	$17.02$	$22.33$	$1.90$	$16.93$	$15.03$
STaSy	$11.29\pm 0.06$	$5.77\pm 0.06$	$9.37\pm 0.09$	$6.29\pm 0.13$	$6.71\pm 0.03$	$7.89$
CoDi	$21.38\pm 0.06$	$15.77\pm 0.07$	$31.84\pm 0.05$	$11.56\pm 0.26$	$16.94\pm 0.02$	$19.50$
TabDDPM	$1.75\pm 0.03$	$1.57\pm 0.08$	$2.72\pm 0.13$	$1.01\pm 0.09$	$1.30\pm 0.03$	$1.67$

Appendix

Table 11: Performance comparison on the error rates (%) of Shape

\downarrow

. ( B : best overall; B : best in language-based model)

Method	Adult	Default	Shoppers	Magic	Beijing	Average
CTGAN	$20.23\pm 1.20$	$26.95\pm 0.93$	$13.08\pm 0.16$	$7.00\pm 0.19$	$22.95\pm 0.08$	$18.04$
TVAE	$14.15\pm 0.88$	$19.50\pm 0.95$	$18.67\pm 0.38$	$5.82\pm 0.49$	$18.01\pm 0.08$	$15.23$
GOGGLE	$45.29$	$21.94$	$23.90$	$9.47$	$45.94$	$29.31$
STaSy	$14.51\pm 0.25$	$5.96\pm 0.26$	$8.49\pm 0.15$	$6.61\pm 0.53$	$8.00\pm 0.10$	$8.71$
CoDi	$22.49\pm 0.08$	$68.41\pm 0.05$	$17.78\pm 0.11$	$6.53\pm 0.25$	$7.07\pm 0.15$	$24.46$
TabDDPM	$3.01\pm 0.25$	$4.89\pm 0.10$	$6.61\pm 0.16$	$1.70\pm 0.22$	$2.71\pm 0.09$	$3.78$

Appendix

Table 12: Performance comparison on the error rates (%) of Trend

\downarrow

.( B : best overall; B : best in language-based model)

Methods	Adult	Default	Shoppers	Magic	Beijing	Average Gap
Methods	AUC $\uparrow$	AUC $\uparrow$	AUC $\uparrow$	AUC $\uparrow$	RMSE $\downarrow$	$\%$
Real	$.927\pm.000$	$.770\pm.005$	$.926\pm.001$	$.946\pm.001$	$.423\pm.003$	$0.0$
CTGAN	$.886\pm.002$	$.696\pm.005$	$.875\pm.009$	$.855\pm.006$	$.902\pm.019$	$28.48$
TVAE	$.878\pm.004$	$.724\pm.005$	$.871\pm.006$	$.887\pm.003$	$.770\pm.011$	$21.09$
GOGGLE	$.778\pm.012$	$.584\pm.005$	$.658\pm.052$	$.654\pm.024$	$1.09\pm.025$	$51.54$
STaSy	$.906\pm.001$	$.752\pm.006$	$.914\pm.005$	$.934\pm.003$	$.656\pm.014$	$12.45$

Appendix

Table 13: Evaluation of MLE. AUC is used for classification tasks and RMSE for regression tasks. ( B : best overall; B : best in language-based model)

Method	Adult	Default	Shoppers	Magic	Beijing	Average
CTGAN	$77.74\pm 0.15$	$62.08\pm 0.08$	$76.97\pm 0.39$	$86.90\pm 0.22$	$96.27\pm 0.14$	$79.99$
TVAE	$98.17\pm 0.17$	$85.57\pm 0.34$	$58.19\pm 0.26$	$86.19\pm 0.48$	$97.20\pm 0.10$	$85.06$
GOGGLE	$50.68$	$68.89$	$86.95$	$90.88$	$88.81$	$77.24$
STaSy	$82.87\pm 0.26$	$90.48\pm 0.11$	$89.65\pm 0.25$	$86.56\pm 0.19$	$89.16\pm 0.12$	$87.74$
CoDi	$77.58\pm 0.45$	$82.38\pm 0.15$	$94.95\pm 0.35$	$85.01\pm 0.36$	$98.13\pm 0.38$	$87.61$
TabDDPM	$96.36\pm 0.20$	$97.59\pm 0.36$	$88.55\pm 0.68$	$98.59\pm 0.17$	$97.93\pm 0.30$	$95.80$

Appendix

Table 14: Evaluation on

\alpha

-Precision scores, ( B : best overall; B : best in language-based model)

Method	Adult	Default	Shoppers	Magic	Beijing	Average
CTGAN	$0.5949$	$0.4875$	$0.7488$	$0.6728$	$0.7531$	$0.6514$
TVAE	$0.6315$	$0.6547$	$0.2962$	$0.7706$	$0.8659$	$0.6438$
GOGGLE	$0.1114$	$0.5163$	$0.1418$	$0.9526$	$0.4779$	$0.4400$
STaSy	$0.4054$	$0.6814$	$0.5482$	$0.6939$	$0.7922$	$0.6242$
CoDi	$0.2077$	$0.4595$	$0.2784$	$0.7206$	$0.7177$	$0.4768$
TabDDPM	$0.9755$	$0.9712$	$0.8349$	0.9998	$0.9513$	$0.9465$

Appendix

Table 15: Evaluation on C2ST, ( B : best overall; B : best in language-based model)

Method	Adult	Default	Shoppers	Beijing	Average $\downarrow$
STaSy	$50.33\%\pm 0.19$	$50.23\%\pm 0.09$	$51.53\%\pm 0.16$	$50.59\%\pm 0.29$	$0.67\%$
CoDi	$\mathbf{49.92\%\pm 0.18}$	$51.82\%\pm 0.26$	$51.06\%\pm 0.18$	$50.87\%\pm 0.11$	$0.96\%$
TabDDPM	$51.14\%\pm 0.18$	$52.15\%\pm 0.20$	$63.23\%\pm 0.25$	$80.11\%\pm 2.68$	$11.66\%$
TabSyn	$50.94\%\pm 0.17$	$51.20\%\pm 0.18$	$52.90\%\pm 0.22$	$50.37\%\pm 0.13$	$1.35\%$
TabDiff	$50.10\%\pm 0.32$	$51.11\%\pm 0.36$	$\mathbf{50.24\%\pm 0.62}$	$50.50\%\pm 0.36$	$0.39\%$
TabDLM	$50.73\%$	$\mathbf{49.95\%}$	$50.87\%$	$\mathbf{50.13\%}$	$\mathbf{0.36\%}$

Appendix

Table 16: Evaluation on DCR score, where a score closer to

50\%

is more preferable. We also report the average absolute deviation of the DCR score from

50\%

across all datasets, where lower values are better ( B : best overall).

Synthetic Tabular Data Generation

Synthetic tabular data generation creates artificial records that preserve useful feature relationships while limiting exposure of sensitive data.

See all 98 papers

Generating Benchmark Health Data Using a Tabular Diffusion Transformer
Aug 14, 2026Hao Yan, Lisa Pilgram, Dan Liu +3Synthetic Tabular Data Generation Tabular Diffusion Models
Children’s Hospital of Eastern Ontario Research Institute, Ontario, Canada · School of Epidemiology and Public Health, University of Ottawa, Ontario, Canada · Department of Nephrology and Medical Intensive Care, Charité - Universitaetsmedizin Berlin, Berlin, Germany +2
ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning
Mar 11, 2026Xiaofeng Lin, Seungbae Kim, Zhuoya Li +3Synthetic Tabular Data Generation Tabular Data Synthesis
University of California, Los Angeles, USA · University of South Florida, USA · Cisco, USA +1
CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data
Sep 30, 2026Mohamed Amine Ketata, Maximilian Schambach, Stephan GünnemannMasked Diffusion Models Tabular Diffusion Models
Munich Data Science Institute, Technical University of Munich, Germany · SAP SE, Germany

Aug 14, 2026J/K move · Enter open · S save

Generating Benchmark Health Data Using a Tabular Diffusion Transformer

Hao Yan, Lisa Pilgram, Dan Liu, Linglong Kong+2

Children’s Hospital of Eastern Ontario Research Institute, Ontario, Canada · School of Epidemiology and Public Health, University of Ottawa, Ontario, Canada · Department of Nephrology and Medical Intensive Care, Charité - Universitaetsmedizin Berlin, Berlin, Germany+2

Synthetic Tabular Data Generation Tabular Diffusion Models

Open paper AskPDF arXiv

Cross-Tabular Data Generation (CTDG) seeks to learn a generative model from multiple heterogeneous tables and produce new synthetic tabular datasets. However, existing synthetic tabular data generation methods are largely restricted to single-input-table scenarios and struggle to effectively handle multiple heterogeneous tables with diverse feature sets. To address this limitation, we propose a two-stage framework for cross-tabular data generation. In the first stage, each heterogeneous raw table is transformed into a standardized statistical table with the same set of columns across all tables. Each statistical table captures the marginal distributions of the original columns and the pairwise correlations among them. In the second stage, a diffusion transformer model is trained to capture structural patterns across these homogeneous statistical tables and to generate synthetic statistical tables. Synthetic raw tables are subsequently reconstructed from the generated statistical tables via multivariate Gaussian sampling followed by an inverse probability integral transform. This two-stage CTDG framework enables the learning of a unified generative model from multiple heterogeneous tables and supports the generation of an unlimited number of realistic synthetic heterogeneous tables. Experimental results demonstrate high fidelity in the learned statistical representations and a favorable fidelity-diversity trade-off in the generated synthetic data, validating the effectiveness of the proposed approach.

Abstract

Synthetic Tabular Data Generation

Explore similar work

Generating Benchmark Health Data Using a Tabular Diffusion Transformer

ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning

CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data