TabGift: A Context-Efficient Foundation Model for Tabular Data Generation

Yu-Sheng Li; Jun-Peng Jiang; Han-Jia Ye

cs.LGOct 12, 2026

TabGift: A Context-Efficient Foundation Model for Tabular Data Generation

Authors: Yu-Sheng Li, Jun-Peng Jiang, Han-Jia Ye

Organizations: School of Artificial Intelligence, Nanjing University, China · National Key Laboratory for Novel Software Technology, Nanjing University, China

Abstract

Tabular data generation aims to synthesize realistic samples from limited observations while faithfully capturing the underlying data distribution. Tabular foundation models (TFMs), pretrained on diverse tabular tasks and adapted through in-context learning, have demonstrated strong predictive capabilities, suggesting the potential of extending this paradigm to tabular data generation. However, the strong predictive performance of TFMs often relies on sufficiently large context sets, whereas tabular generation is typically needed precisely when target data are scarce. Extending the foundation-model paradigm to generation therefore requires reliable distribution inference from only a limited context. To this end, we propose TabGift, a context-efficient tabular foundation model enabling both efficient distribution modeling and effective generation from only a few context samples. TabGift maps heterogeneous tables into a shared latent space with a pretrained tabular encoder, where an in-context diffusion model learns to denoise latent samples conditioned on a small set of clean examples. This design transfers generative priors across datasets and enables distribution inference from limited context, while a lightweight dataset-specific MLP decoder maps generated representations back to the original data space. Experiments on 7 classification and 11 regression datasets show that TabGift consistently outperforms representative baselines in synthetic data quality, with particularly strong gains under limited-context settings, while achieving substantially faster adaptation and generation.

Figures & tables

Figure 1: TabGift consists of a data space and a latent space: (1) Encoding: Given the real training table $X_{\text{train}}$ , the SwiftPFN Encoder maps heterogeneous samples into a unified 192-dimensional latent space, producing the latent embeddings $Z_{\text{train}}$ . (2) Latent-space generation: The real-data latent latent embedding $Z_{\text{train}}$ is used as context. Starting from Gaussian noise, a pretrained frozen denoising model generates new latent embeddings $\hat{Z}_{\text{syn}}$ through an iterative denoising process. (3) Decoding: A lightweight MLP Decoder is trained per target dataset to reconstruct the generated latent embeddings $\hat{Z}_{\text{syn}}$ back into the data space, yielding synthetic data $\hat{X}_{\text{syn}}$ . Throughout the pipeline, only the MLP Decoder requires dataset-specific training, while the SwiftPFN Encoder and the denoising model are pretrained and kept frozen during inference.

Figure 3: Loss curves during the two-stage training. The light blue curve shows the raw loss, and the dark blue curve shows the loss smoothed by a moving average with a window size of 10.

Classification (gen/real, higher is better)
Method	16	32	64	128	256	512
Real	$1.0000_{\pm 0.0000}$	$1.0000_{\pm 0.0000}$	$1.0000_{\pm 0.0000}$	$1.0000_{\pm 0.0000}$	$1.0000_{\pm 0.0000}$	$1.0000_{\pm 0.0000}$
SMOTE	$0.9927_{\pm 0.0329}$	$0.9885_{\pm 0.0456}$	$0.9990_{\pm 0.0249}$	$0.9979_{\pm 0.0184}$	$0.9997_{\pm 0.0171}$	$0.9980_{\pm 0.0104}$
TVAE	$1.0035_{\pm 0.0933}$	$0.9761_{\pm 0.1022}$	$0.9502_{\pm 0.0926}$	$0.9368_{\pm 0.0574}$	$0.9310_{\pm 0.0772}$	$0.8988_{\pm 0.0685}$
CTGAN	$0.9918_{\pm 0.1292}$	$0.9611_{\pm 0.1227}$	$0.9231_{\pm 0.1022}$	$0.9019_{\pm 0.1083}$	$0.9162_{\pm 0.0976}$	$0.8842_{\pm 0.1055}$
TabDDPM	$0.9533_{\pm 0.0966}$	$0.9046_{\pm 0.1158}$	$0.9306_{\pm 0.1159}$	$0.9176_{\pm 0.1201}$	$0.9028_{\pm 0.1721}$	$0.8925_{\pm 0.1348}$

Table 1: Relative performance (mean

\pm

std) of different generation methods under various training sizes. For classification, we report

D_{\mathrm{syn}}/D_{\mathrm{real}}

(gen/real, higher is better); for regression, we report

D_{\mathrm{real}}/D_{\mathrm{syn}}

(real/gen, higher is better). Standard deviations are shown as subscripts. The best , second-best , and third-best results among generative methods are highlighted. Real is shown as a reference and is not ranked.

Figure 4: Synthetic generator comparison on statistical fidelity (Inv KL, KS test) and privacy protection (DCR, Delta Presence). Shown for training size 16 across all datasets, mean over 5 seeds.

Method	16			32			64			128			256			512
Method	Train	Infer	Total	Train	Infer	Total	Train	Infer	Total	Train	Infer	Total	Train	Infer	Total	Train	Infer	Total
SMOTE	0.000	0.010	0.010	0.000	0.009	0.009	0.000	0.009	0.009	0.000	0.013	0.013	0.000	0.011	0.011	0.000	0.014	0.014
TVAE	17.316	0.161	17.477	13.607	0.168	13.775	17.950	0.148	18.098	17.986	0.197	18.182	33.206	0.130	33.336	54.554	0.167	54.721
CTGAN	30.704	0.128	30.832	32.919	0.129	33.049	50.764	0.113	50.878	55.871	0.101	55.972	69.029	0.095	69.123	94.568	0.097	94.665
TabDDPM	29.110	333.780	362.890	29.493	135.185	164.677	28.566	71.906	100.472	26.288	67.191	93.479	24.295	20.538	44.834	26.733	19.629	46.362
ARF	1.950	3.535	5.486	1.484	3.156	4.640	1.712	3.276	4.988	1.858	2.802	4.660	2.925	3.299	6.224	4.484	2.853	7.337

Table 2: Average runtime (seconds, mean) of different generation methods under various training sizes. For each size, we report training time, inference time, and total time.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Features	#Samples	#Cat / #Num
abalone	8	4,177	1 / 7
airfoil_self_noise	5	1,503	0 / 5
auction_verification	7	2,043	6 / 1
concrete_compressive_strength	8	1,030	0 / 8
physiochemical_protein	9	45,730	0 / 9
superconductivity	81	21,263	2 / 79

Appendix

Table 3: Statistics of the OpenML regression datasets.

Dataset	#Classes	#Features	#Samples	#Cat / #Num	Dataset	#Classes	#Features	#Samples	#Cat / #Num
kr-vs-kp	2	36	3,196	36 / 0	pc1	2	21	1,109	0 / 21
letter	26	16	20,000	16 / 0	adult	2	14	48,842	9 / 5
balance-scale	3	4	625	0 / 4	Bioresponse	2	1,776	3,751	834 / 942
mfeat-factors	10	216	2,000	22 / 194	wdbc	2	30	569	0 / 30
mfeat-fourier	10	76	2,000	0 / 76	phoneme	2	5	5,404	0 / 5
breast-w	2	9	699	0 / 9	qsar-biodeg	2	41	1,055	22 / 19

Appendix

Table 4: Statistics of the OpenML classification datasets.

Dataset	#Rows	#Features	#Cat	#Num	#Classes
FOREX_audchf-day-High	1,833	10	0	10	2
FOREX_audsgd-hour-High	43,825	10	0	10	2
FOREX_cadjpy-hour-High	43,825	10	0	10	2
Gender_Gap_in_Spanish_WP	4,746	13	0	13	3
adult	45,222	14	8	6	2
moons	512	2	0	2	2

Appendix

Table 5: Statistics of the TALENT classification datasets.

Dataset	#Rows	#Features	#Cat	#Num	#Unique Targets
Ailerons	13,750	40	12	28	35
CookbookReviews	18,182	7	1	6	6
IEEE80211aa-GATS	4,046	27	2	25	3,407
Large-scale_Wave_Energy_Farm_Sydney_49	17,964	99	0	99	4,816
Superconductivty	21,197	81	2	79	3,007
VulNoneVul	5,692	16	0	16	2

Appendix

Table 6: Statistics of the TALENT regression datasets.

Synthetic Tabular Data Generation

Synthetic tabular data generation creates artificial records that preserve useful feature relationships while limiting exposure of sensitive data.

See all 98 papers

TabFORGE: A Tabular Foundation Model for Generative Modelling
May 10, 2026Xiangjian Jiang, Mingxuan Liu, Nikola Simidjievski +2Tabular Foundation Models Synthetic Tabular Data Generation
Department of Computer Science and Technology, University of Cambridge, UK · SAP SE · Télécom Paris, Institut Polytechnique de Paris, France
ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning
Mar 11, 2026Xiaofeng Lin, Seungbae Kim, Zhuoya Li +3Synthetic Tabular Data Generation Tabular Data Synthesis
University of California, Los Angeles, USA · University of South Florida, USA · Cisco, USA +1
TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity
Oct 6, 2026Zijian Li, Xiangchen Song, Gongxu Luo +7Synthetic Data Pretraining Tabular Foundation Models
Carnegie Mellon University · Mohamed bin Zayed University of Artificial Intelligence · Guangdong University of Technology +1

May 10, 2026J/K move · Enter open · S save

TabFORGE: A Tabular Foundation Model for Generative Modelling

Xiangjian Jiang, Mingxuan Liu, Nikola Simidjievski, Tassilo Klein+1

Department of Computer Science and Technology, University of Cambridge, UK · SAP SE · Télécom Paris, Institut Polytechnique de Paris, France

Tabular Foundation Models Synthetic Tabular Data Generation Tabular Representation Learning

Open paper AskPDF arXiv

Generative modelling is a demanding test of foundation models, because it requires robust, holistic representation learning for a given data modality, rather than optimisation for a supervised prediction target alone. While recent work on tabular foundation models has achieved remarkable progress in predictive modelling, generative tabular foundation models remain underexplored. Existing tabular foundation generators, in particular, have not yet consistently matched strong dataset-specific generators in synthetic data quality. A key reason is their insufficient alignment with the inter-feature dependency structure of tabular data, which can reflect underlying causal relationships. In this paper, we address this gap by introducing a novel tabular foundation model,

\textbf{TabFORGE}

, built on pretrained

\textbf{Tab}

ular

\textbf{FO}

undational

\textbf{R}

epresentations for

\textbf{GE}

neration. TabFORGE is designed to utilise transferable representations induced by a pretrained structure-aware feature encoder, which captures information useful for characterising inter-feature dependencies. It further decouples latent modelling from decoding through a two-stage design: we first pretrain a score-based diffusion transformer, and then pretrain a denoising-aligned decoder using the denoised latent embeddings. This design elegantly mitigates the distribution shifts in latent embeddings that typically arise between training and inference. We evaluate TabFORGE comprehensively against 22 benchmark methods on 45 real-world datasets. Our results show that TabFORGE effectively learns and leverages generalisable tabular representations, enabling efficient generation of high-quality synthetic tabular data, particularly with strong structural fidelity. Code is available at https://github.com/SilenceX12138/TabFORGE.

Abstract

Synthetic Tabular Data Generation

Explore similar work

TabFORGE: A Tabular Foundation Model for Generative Modelling

ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning

TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity