TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks

Jayoung Kim; Sanghyun Hong; Noseong Park

cs.LGOct 11, 2026

TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks

Authors: Jayoung Kim, Sanghyun Hong, Noseong Park

Organizations: KAIST · Oregon State University

Abstract

Generative modeling for tabular data has advanced to the point where models can synthesize realistic records, recover missing values, and generate samples conditioned on class labels or observed attributes. Despite recent advances, existing approaches either require table-specific training or restrict in-context learning to single-cell prediction. In this work, we formulate these tabular generative tasks as a single problem: modeling the joint conditional distribution of an arbitrary set of target cells given the observed cells of an unseen table. To this end, we turn to diffusion, whose denoising objective is defined jointly over all target cells rather than one cell at a time. Under the pretraining distribution, this objective is minimized by the posterior predictive distribution over the target set. Building on this, we propose TabMosaic, a unified in-context diffusion model for generating mixed-type target cells in unseen, potentially incomplete tables without table-specific training. We further show that its pretraining objective bounds the conditional negative log-likelihood and the divergence from the posterior predictive for arbitrary target sets, extending the Bayesian account of in-context learning from a single-value prediction to joint multi-cell generation. Across 48 real-world tables, TabMosaic ranks first on all evaluated generative tasks against 20 baseline approaches

Figures & tables

Figure 1: Overview of our proposed framework. (a) A unified formulation expresses generation, imputation and class-conditional generation as the same conditional query, only differing in the target set

S

. (b) A shared-time diffusion objective jointly models mixed numerical and categorical targets. (c) Our TabMosaic applies directly to unseen tables without table-specific training.

Figure 2: Elo with

95\%

bootstrap intervals for generation and imputation on the

48

tables and class-conditional generation on the

36

classification tables. Bar color marks default and tuned entries, name color the model family, and hatching (

\dagger

) a method that did not complete every table, whose missing tables are scored as described in Appendix G . Full results are in Appendix L .

Figure 3: NLL of held-out cells against the context size

n

, for a small model pretrained with Eq. 6 on priors with a closed-form posterior predictive (

x

: numerical,

c

: categorical cell; Appendix I ). Solid black: exact posterior predictive; dashed:

M

exact samples under the same KDE as the model, i.e., the floor for a sampler. Shaded: beyond the context sizes seen in pretraining.

Figure 4: Time to generate one synthetic table against the number of rows and of columns (median of three runs).

\times

: over the one-hour limit. Reds: per-table diffusion; oranges: per-table autoregressive; blues: in-context predictors sampled column by column.

Figure 5: Distance to the exact posterior predictive as the target set

S

grows on the Dirichlet prior. (Appendix I ).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

	Stage 1	Stage 2
Rows $T$	$1{,}024$	$\log\mathcal{U}(400,10{,}240)$
Features $F$	$\log\mathcal{U}(1,100)$	$\log\mathcal{U}(4,\min(200,B/T))$ , $B=250{,}000$
Context fraction	$\mathcal{U}(0.3,0.9)$	$0.8$
Context missingness	—	$\rho\sim\mathcal{U}(0,\rho_{\max})$ , $\rho_{\max}$ : $0\to 0.2$ over $2{,}500$ steps
Corruptions $K$ per table	$1$	$4$
Steps / tables seen	$125$ k / $32$ M	$10$ k / $2.56$ M

Appendix

Table 2: Pretraining configuration. Both stages use

8

GPUs.

Task	$S$	$x_{\mathrm{obs}}$	$t_{\mathrm{start}}$	Sampler (num.) / $N$	$M$ ; output
Generation	all query cells	—	$1$	deterministic / $128$	$1$ ; one row per trajectory
Class-cond. gen.	feature cells	label cell	$1$	deterministic / $128$	$1$ ; one row per trajectory
Imputation	missing cells	observed cells	$0.5$	ancestral / $16$	$8$ ; mean (num.), vote (cat.)

Appendix

Table 3: Settings per task.

t_{\mathrm{start}}

is the time at which the reverse process starts,

N

the number of steps, and

M

the number of trajectories per query row; the context is encoded once per query batch.

Mechanism	Methods	Where the conditional comes from
Pretrained conditional	TabMosaic	label cell observed; the conditional is a training term
Trained conditional, per table	TabNAT	masked training over random orders on the table
Class input at training	TabDDPM	the label is an input of the denoiser; not generated
One model per class	ForestDiffusion	a separate model is trained for each class
Discrete-column condition	CTGAN	condition vector plus a cross-entropy penalty (soft)
Prompt	GReaT	the label appears in the prompt; permutation training

Appendix

Table 4: How each method conditions on the class in class-conditional generation.

Method	$n$	Elo	Utility $\uparrow$	$\alpha$ -prec. $\uparrow$	$\beta$ -rec. $\uparrow$	Auth. $\uparrow$	W 1 $\downarrow$	TV $\downarrow$	$\|\Delta\rho\|$ $\downarrow$	Time (s)
TabMosaic	48	1099 [1063, 1126]	0.914	0.790	0.408	0.569	0.054	0.038	0.038	25
TabNAT	48	1093 [1057, 1121]	0.899	0.805	0.421	0.559	0.046	0.023	0.030	1696
TabSyn	48	1060 [1030, 1079]	0.864	0.738	0.385	0.590	0.059	0.055	0.037	369
TabDiff (t)	48	1059 [1031, 1086]	0.886	0.730	0.380	0.591	0.060	0.022	0.042	2119
TabPFN-3 (chained)	48	1055 [1027, 1088]	0.891	0.662	0.353	0.632	0.079	0.081	0.059	571
ForestDiffusion †	42	1051 [1023, 1081]	0.886	0.770	0.409	0.568	0.059	0.061	0.030	1181

Appendix

Table 5: Generation on the

48

tables: Elo with

95\%

bootstrap intervals and per-metric means over the tables each method completed (

n

). Utility is relative to training on real rows;

\alpha

-precision,

\beta

-recall, and authenticity follow Alaa et al. (2022) ; W 1 (median over tables) and TV are column-level distances and

|\Delta\rho|

the mean absolute difference of pairwise correlations. Time is the median wall-clock per table in seconds (training plus sampling for per-table models; inference for the others), measured on one GPU. The Elo tasks are table

\times

{utility,

\alpha

-precision,

\beta

-recall, authenticity}. (t) = run with the hyperparameters tuned in the original paper, which provides no default.

\dagger

: not completed on every table; missing tables are filled with column resampling in the Elo.

Method	$n$	Elo	Elo (common)	Error $\downarrow$	Rank $\downarrow$	RMSE $\downarrow$ (num.)	Error rate $\downarrow$ (cat.)	Downstream $\uparrow$
TabMosaic	48	1458 [1389, 1521]	1458 [1382, 1539]	0.129	1.88	0.108 (44)	0.314 (33)	0.962
TabICLv2 (per cell)	48	1201 [1168, 1238]	1213 [1167, 1258]	0.146	4.38	0.126 (44)	0.375 (33)	0.839
ReMasker (t)	48	1108 [1071, 1145]	1113 [1075, 1159]	0.157	3.58	0.121 (44)	0.467 (33)	0.829
HyperImpute	48	1073 [1040, 1100]	1071 [1043, 1103]	0.163	5.19	0.129 (44)	0.473 (33)	0.828
ReMasker	48	1066 [1034, 1099]	1072 [1033, 1114]	0.167	4.90	0.133 (44)	0.483 (33)	0.827
DiffPuter	48	1063 [1019, 1104]	1086 [1045, 1125]	0.204 (3 excl.)	7.31	0.170 (44)	0.466 (33)	0.907

Appendix

Table 6: Imputation on the

48

tables (

30\%

of the feature cells hidden completely at random;

3

mask seeds). The Elo tasks are table

\times

{RMSE on numerical cells, error rate on categorical cells, downstream utility} (125 tasks); the second Elo uses only the 41 tables every method completed (107 tasks). Error is the primary score of a table (min-max–scaled RMSE where the table has numerical cells, misclassification rate otherwise), averaged over the tables each method completed (

n

); rank is the mean rank over tables. RMSE and error rate are also reported separately over the tables with numerical and with categorical cells (counts in parentheses); cells with RMSE above

5

(out-of-range outputs) are excluded from the means and counted. Downstream is the utility of a predictor trained on the imputed table relative to one trained on the complete table. (t) = tuned.

\dagger

: not completed on every table; missing tables are filled with the mean/mode imputer in the first Elo.

Method	$n$	Elo	Utility $\uparrow$	$\alpha$ -prec. $\uparrow$	$\beta$ -rec. $\uparrow$	Consist. (rel.)	Consist. (macro)	Agreement	TV $\downarrow$	W 1 $\downarrow$
TabMosaic	36	1163 [1135, 1191]	0.926	0.624	0.349	0.983	0.675	1.00	0.064	0.096
TabPFN-3 (chained)	36	1144 [1116, 1170]	0.917	0.591	0.349	0.986	0.676	1.00	0.086	0.103
TabNAT	36	1122 [1091, 1157]	0.881	0.675	0.381	0.972	0.671	1.00	0.048	0.117
ForestDiffusion †	32	1076 [1040, 1109]	0.908	0.638	0.386	1.016	0.738	1.00	0.081	0.093
TabEBM †	32	1053 [1015, 1089]	0.922	0.526	0.363	1.008	0.723	1.00	0.078	0.073
TabICLv2 (chained)	36	1019 [991, 1048]	0.911	0.449	0.267	0.941	0.665	1.00	0.084	0.367

Appendix

Table 7: Class-conditional generation on the

36

classification tables: Elo with

95\%

bootstrap intervals and means over the tables each method completed (

n

). The Elo tasks are table

\times

{utility, class-macro

\alpha

-precision, class-macro

\beta

-recall, label-consistency gap}, where the gap is the absolute difference between the label consistency of the synthetic rows and that of held-out real rows. Label consistency is the fraction of synthetic rows assigned to the requested class by the scoring predictors trained on real rows, divided by the same fraction on held-out real rows (

1

= as separable as real data; above

1

= classes separated more than in the real data); macro is the unnormalized macro-average over classes; agreement is the fraction of rows whose emitted label equals the requested class. TV and W 1 (median) are per-class distances, macro-averaged. (t) = the generation model with the hyperparameters tuned in its original paper.

\dagger

: not completed on every table; a missing table is scored just below the worst method on that table in the Elo.

		TabMosaic	TabNAT	ForestDiff.	DiffPuter	GAIN	MICE	TabPFN-3	TabICLv2	LimiX
MCAR rate	$10\%$	0.622	0.777	1.000	0.901	1.001	1.157	0.912	0.729	0.872
	$30\%$	0.675	0.860	1.082	0.966	1.028	1.233	1.098	0.812	0.916
	$50\%$	0.770	0.932	1.113	0.955	1.141	1.237	1.212	0.921	0.966
	$70\%$	0.843	0.993	1.182	1.005	1.290	1.296	1.328	0.974	1.016
$30\%$	MAR	0.628	0.887	1.065	0.887	1.064	1.212	1.019	0.807	0.931
$30\%$	MNAR	0.704	0.871	1.012	0.866	1.063	1.143	1.068	0.824	0.910

Appendix

Table 8: Per-cell error on held-out rows relative to a mean/mode imputer (median over tables; below

1

= better than mean/mode), as the target set varies: the MCAR rate, the mechanism at

30\%

, and the number

|S|

of whole hidden columns.

24

tables on which every method completed (the

15

smallest and

9

medium-sized ones;

23

for

|S|{=}4

). Best per row in bold.

Synthetic Tabular Data Generation

Synthetic tabular data generation creates artificial records that preserve useful feature relationships while limiting exposure of sensitive data.

See all 98 papers

TabGift: A Context-Efficient Foundation Model for Tabular Data Generation
Oct 12, 2026Yu-Sheng Li, Jun-Peng Jiang, Han-Jia YeSynthetic Tabular Data Generation Generative Modeling
School of Artificial Intelligence, Nanjing University, China · National Key Laboratory for Novel Software Technology, Nanjing University, China
TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion
Feb 26, 2026Donghong Cai, Jiarui Feng, Yanbo Wang +3Synthetic Tabular Data Generation Tabular Diffusion Models
Washington University in St. Louis · Peking University · Ant Group
MIND: Marginal-Invariant Neural Dependency Diffusion for Mixed-Type Tabular Generation
Sep 30, 2026Pengfei Li, Mohammad KhalilCopula Models Synthetic Tabular Data Generation
Centre for the Science of Learning & Technology (SLATE), University of Bergen, Bergen, Norway

Oct 12, 2026J/K move · Enter open · S save

TabGift: A Context-Efficient Foundation Model for Tabular Data Generation

Yu-Sheng Li, Jun-Peng Jiang, Han-Jia Ye

School of Artificial Intelligence, Nanjing University, China · National Key Laboratory for Novel Software Technology, Nanjing University, China

Synthetic Tabular Data Generation Generative Modeling Tabular Foundation Models Tabular Diffusion Models Tabular ICL

Open paper AskPDF arXiv

Tabular data generation aims to synthesize realistic samples from limited observations while faithfully capturing the underlying data distribution. Tabular foundation models (TFMs), pretrained on diverse tabular tasks and adapted through in-context learning, have demonstrated strong predictive capabilities, suggesting the potential of extending this paradigm to tabular data generation. However, the strong predictive performance of TFMs often relies on sufficiently large context sets, whereas tabular generation is typically needed precisely when target data are scarce. Extending the foundation-model paradigm to generation therefore requires reliable distribution inference from only a limited context. To this end, we propose TabGift, a context-efficient tabular foundation model enabling both efficient distribution modeling and effective generation from only a few context samples. TabGift maps heterogeneous tables into a shared latent space with a pretrained tabular encoder, where an in-context diffusion model learns to denoise latent samples conditioned on a small set of clean examples. This design transfers generative priors across datasets and enables distribution inference from limited context, while a lightweight dataset-specific MLP decoder maps generated representations back to the original data space. Experiments on 7 classification and 11 regression datasets show that TabGift consistently outperforms representative baselines in synthetic data quality, with particularly strong gains under limited-context settings, while achieving substantially faster adaptation and generation.

Abstract

Synthetic Tabular Data Generation

Explore similar work

TabGift: A Context-Efficient Foundation Model for Tabular Data Generation

TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion

MIND: Marginal-Invariant Neural Dependency Diffusion for Mixed-Type Tabular Generation