TabMF: Learning Dataset Representations with Tabular Foundation Models

Rafiq Kamel; Nicolas Atienza; Aziz Fouché; Nicola Cancedda; Josif Grabocka

cs.LGOct 9, 2026

TabMF: Learning Dataset Representations with Tabular Foundation Models

Authors: Rafiq Kamel, Nicolas Atienza, Aziz Fouché, Nicola Cancedda, Josif Grabocka

Organizations: University of Technology Nuremberg (UTN) · Neuralk

Abstract

Representations of supervised datasets support higher-level machine-learning tasks such as dataset retrieval and distribution-shift monitoring. However, existing approaches typically rely either on hand-crafted meta-features or on dataset encoders trained specifically for representation learning. We introduce TabMF, a framework that repurposes pretrained tabular foundation models (TFMs) as encoders of supervised datasets. TabMF equips a predictive TFM with an architecture-aware dataset-level readout and adapts the model end-to-end using contrastive post-training on stochastic views of synthetic supervised datasets. We instantiate TabMF on TabICLv2 and TabPFN-2.5 and evaluate the resulting representations across six tasks on unseen real datasets. Simple frozen TFM readouts yield competitive representations, indicating that predictive pretraining already induces substantial dataset-level structure. The two TabMF encoders obtain the strongest aggregate performance among the evaluated learned and classical dataset representations, with particularly strong results in dataset retrieval, compact benchmark construction, and distribution-shift severity assessment.

Figures & tables

Panel of Figure 1 — Figure 1: TabMF overview. Left: A synthetic supervised dataset is sampled from one Graph-SCM, and two stochastic views vary the sampled observations, observed variables, and label-preserving feature transformations while sharing the same designated target. TabMF encodes both views and is trained contrastively to bring their representations together while separating representations of other datasets. Right: At inference time, a frozen TabMF encoder maps a new supervised dataset $(X,y)$ to a fixed-dimensional representation that can be used directly for dataset comparison and monitoring or as input to lightweight downstream probes.

Encoder	Views	Domain	Epiplexity	Recall@5	Selection	Severity	Rank $\downarrow$
	TabBench				TabRepo	TableShift
TabMF -PFN	0.9985	0.683	0.597	0.965	0.929	0.917	1.5
TabMF -ICL	0.9981	0.702	0.639	0.938	0.854	0.933	2.0
TabICL ICL-mean	0.9775	0.670	0.742	0.889	0.575	0.850	3.5
PyMFE	0.9812	0.661	0.106	0.931	0.709	0.733	4.0
TabPFN mean-pooling	0.9414	0.650	0.741	0.821	0.850	0.583	5.0

Table 1: Primary results across six evaluation tasks. Scores are real-data bootstrap AUC, domain macro-OVR AUC, epiplexity regression Spearman, mean retrieval recall@5, TabRepo three-dataset K-means ranking Spearman, and TableShift shift-severity Spearman. Encoders are ordered by their median rank across the six tasks. Higher scores and lower ranks are better; bold marks numerical leaders. Epiplexity uses available-case scores and leakage-controlled PyMFE features; paired common-population results are in Appendix B.4.3 .

Panel of Figure 2 — Figure 2: Dataset identity remains recoverable with incomplete features, but robustness depends on the encoder and transformation. ( 2(a) ) Both TabMF variants outperform ICL-mean and PyMFE when only 25% of columns remain. ( 2(b) ) TabMF -PFN better tolerates monotone transformations and standardization, while TabMF -ICL better tolerates heavy noise and missingness; quantization limits both variants. Values are retrieval recall@5; the complete transformation results appear in Figure 3 .

Backbone	Readout	Initialization	Objective	Recall@5
TabICL	Summary-token	Pretrained	NT-Xent	0.938
TabICL	Summary-token	Scratch	NT-Xent	0.797
TabICL	Summary-token	Pretrained	Triplet	0.924
TabICL	Summary-token	Pretrained	Exp. distance	0.883
TabPFN	Column-attention	Pretrained	NT-Xent	0.965
TabPFN	Column-attention	Scratch	NT-Xent	0.587

Table 2: Encoder design comparisons using mean retrieval recall@5. Each block compares configurations within one backbone across initialization, readout, and objective choices.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Backbone	Readout	Loss	Initialization	AUC
TabICL	Summary-token	NT-Xent	Pretrained	0.9999
TabPFN	Column-aware	NT-Xent	Pretrained	0.9996
TabICL	Row-mean	NT-Xent	Pretrained	0.9996
TabPFN	Pseudo-test	NT-Xent	Pretrained	0.9995
TabICL	Row-mean	Triplet	Pretrained	0.9994
TabPFN	Pseudo-test	Triplet	Pretrained	0.9994

Appendix

Table 3: Synthetic bootstrap-view discrimination AUC across encoder configurations. Row-mean denotes a trained encoder, distinct from the frozen row-mean baseline.

Population	Datasets	Significant improvements	Largest Holm $p$
TabBench	191	18/18	$<1.6\times 10^{-16}$
Synthetic	50	18/18	$<5.0\times 10^{-14}$

Appendix

Table 4: Paired comparisons of per-dataset hard margins in bootstrap-view discrimination. Both adapted encoders improve over each of nine baselines in each population. Holm correction is applied across all 18 comparisons within each population. The final column gives an upper bound on the largest adjusted

p

-value. These tests assess margins, not AUC. Synthetic results characterize the population used for model selection.

Transformation	Definition
Keep 20% / 50% of rows	Sample $k_{r}$ observations without replacement, $r\in\{0.2,0.5\}$ . Skip when $k_{r}\geq n$ .
Bootstrap rows	Sample $n$ observations with replacement.
Duplicate rows	Retain all observations and append $\max\{1,\operatorname{round}(0.3n)\}$ observations sampled with replacement.
Keep 25% / 50% / 75% of features	Sample $h_{r}$ features without replacement, $r\in\{0.25,0.5,0.75\}$ . Skip when $h_{r}\geq d$ .
Duplicate features	Append copies of $h_{0.3}$ features selected without replacement.
Add noisy features	Append $h_{0.3}$ numerical features with independent $\mathcal{N}(0,1)$ entries.

Appendix

Table 5: Dataset transformations used for retrieval evaluation. Here,

n

and

d

denote the original numbers of observations and features, and

\mu_{j}

and

\sigma_{j}

denote the mean and standard deviation of numerical feature

j

. We write

k_{r}=\max\{\min(n,20),\operatorname{round}(rn)\}

and

h_{r}=\max\{1,\operatorname{round}(rd)\}

. Grouped entries denote separate transformations for each listed parameter value.

Baseline	TabMF -ICL	TabMF -PFN
TabICL ICL-mean	$\mathbf{1.0\times 10^{-7}}$	$\mathbf{5.8\times 10^{-15}}$
PyMFE	$0.522$	$\mathbf{1.6\times 10^{-5}}$

Appendix

Table 6: Dataset-level paired comparisons of transformation retrieval. For each dataset, recall@5 is averaged across applicable transformations. Entries are Holm-adjusted

p

-values from two-sided Wilcoxon signed-rank tests, correcting across all four comparisons. Tests pair 190 datasets, except TabMF -PFN versus PyMFE, which pairs 191. Bold denotes a significant improvement of the adapted encoder. These tests assess dataset-level averages; the headline retrieval score instead averages recall@5 equally across transformations.

Encoder	K-means	Farthest-first
TabMF -PFN	0.9292	0.8229
TabMF -ICL	0.8542	0.8146
TabPFN mean-pooling	0.8500	0.6730
Random	0.7766	0.7766
Neural Statistician	0.738	0.782
PyMFE	0.7093	0.7812

Appendix

Table 7: Compact benchmark selection on TabRepo at

k=3

. Entries are mean classifier-ranking Spearman correlations over 200 shared candidate-pool trials. Encoders are ordered by K-means performance. Bold marks numerical leaders.

Variant	Baseline	$\Delta\rho$	Holm $p$
TabMF -ICL	Random	$+0.0777$	$5.28\times 10^{-7}$
TabMF -ICL	ICL-mean	$+0.2792$	$6.03\times 10^{-28}$
TabMF -ICL	PyMFE	$+0.1450$	$2.67\times 10^{-22}$
TabMF -ICL	Dataset2Vec	$+0.2028$	$1.74\times 10^{-25}$
TabMF -ICL	Neural Statistician	$+0.1162$	$2.00\times 10^{-19}$
TabMF -ICL	Frozen TabPFN	$+0.0045$	$0.2408$

Appendix

Table 8: K-means benchmark-selection comparisons on TabRepo at a three-dataset budget. Differences are TabMF minus baseline in mean Spearman correlation. Two-sided Wilcoxon signed-rank tests pair the 200 candidate-pool trials. Reported

p

-values are Holm-adjusted jointly over the 12 comparisons.

Variant	Baseline	Original	Alternative
TabMF -ICL	ICL-mean	$+0.2792$	$+0.2602$
TabMF -ICL	PyMFE	$+0.1450$	$+0.1280$
TabMF -ICL	Dataset2Vec	$+0.2028$	$+0.1946$
TabMF -PFN	ICL-mean	$+0.3542$	$+0.3478$
TabMF -PFN	PyMFE	$+0.2199$	$+0.2155$
TabMF -PFN	Dataset2Vec	$+0.2777$	$+0.2822$

Appendix

Table 9: Sensitivity of three-dataset selection on TabRepo to K-means initialization. Entries are paired mean-correlation differences ( TabMF minus baseline). Candidate pools and embeddings are fixed. The alternative initialization preserves the direction of all six comparisons.

$k$	Selector	TabMF -ICL	TabMF -PFN	ICL-mean	PyMFE	Random	D2V	Frozen PFN	Neural Stat.
2	K-means	0.8568	0.6728	0.3507	0.6778	0.7209	0.6947	0.8214	0.7828
2	Farthest-first	0.7003	0.8752	0.5941	0.6979	0.7209	0.7100	0.6439	0.6418
3	K-means	0.8542	0.9292	0.5750	0.7093	0.7766	0.6515	0.8497	0.7380
3	Farthest-first	0.8146	0.8229	0.7108	0.7812	0.7766	0.7261	0.6728	0.7819
4	K-means	0.8504	0.9393	0.6743	0.8894	0.8171	0.7282	0.8536	0.7260
4	Farthest-first	0.8127	0.8697	0.8148	0.8338	0.8171	0.7805	0.7447	0.8560

Appendix

Table 10: TabRepo benchmark selection with two to four selected datasets. Entries are mean Spearman correlations with the full-population classifier ranking. Bold marks the best score within each fixed budget and selector. Various budgets assess sensitivity to the selection budget.

Encoder	AUC $\uparrow$	Log-loss $\downarrow$	$N$
TabMF -ICL	0.6997	1.7860	190
TabMF -PFN	0.6808	1.8100	191
PyMFE	0.6662	1.8871	191
TabICL row-mean	0.6637	1.8850	190
TabPFN mean-pooling	0.6620	1.8535	191
TabICL ICL-mean	0.6441	1.8585	190

Appendix

Table 11: Domain discrimination with the strengthened nonlinear probe. A tuned SVM jointly searches RBF and polynomial kernels. AUC and log-loss are computed on each encoder’s maximal available population. Pairwise inference uses per-dataset log-loss on the population shared by each pair, with Holm correction over all 55 encoder comparisons. Bold denotes the best point estimate.

Encoder	Spearman $\rho$
TabICL ICL-mean	0.764
TabPFN mean-pooling	0.760
TabMF -ICL	0.639
TabMF -PFN	0.621
Dataset2Vec	0.529

Appendix

Table 12: Epiplexity prediction on a common population of 190 datasets. A: Mean Spearman correlation across five cross-validation repetitions. B: Frozen-versus-adapted comparisons from a paired dataset bootstrap, with Holm adjustment over all ten pairwise comparisons among the five encoders. Bold

p

-values support higher correlation for the frozen readout.

Encoder	Mean $\|\mathrm{error}\|$ $\downarrow$	Spearman $\uparrow$	$N$
TabPFN mean-pooling	0.1770	0.8292 $\pm$ 0.026	191
TabMF -PFN	0.1794	0.7633 $\pm$ 0.019	191
TabICL ICL-mean	0.2032	0.7991 $\pm$ 0.021	190
TabMF -ICL	0.2224	0.6953 $\pm$ 0.012	190
Dataset2Vec	0.2265	0.6182 $\pm$ 0.026	191
Neural Statistician	0.3297	0.2658 $\pm$ 0.027	191

Appendix

Table 13: Epiplexity prediction with the strengthened nonlinear probe. Kernel ridge regression jointly searches RBF, Laplacian, and polynomial kernels. Mean absolute error is the endpoint used for ranking and paired inference; Spearman correlation is included for comparison with the primary linear probe. Pairwise tests use the population shared by each pair, with Holm correction over all 55 comparisons. Bold denotes the best point estimate.

Excluded base-feature identifiers
nr_class	freq_class	class_ent
class_conc	mut_inf	joint_ent
eq_num_attr	ns_ratio	can_cor
gravity	lh_trace	p_trace
roy_root	w_lambda	sd_ratio

Appendix

Table 14: Explicit exclusions in the leakage-controlled PyMFE difficulty configuration. Landmarking and model-based groups are omitted entirely. The remaining general, statistical, and information-theoretic groups are fitted without the dataset target. All summaries of each listed base feature are excluded together.

Encoder	TR $k=2$	TR $k=3$	TR $k=4$	Agg. $k=2$	Agg. $k=3$	Agg. $k=4$
TabMF -PFN	6	1	1	2.0	1.5	1.5
TabMF -ICL	1	2	4	1.5	2.0	2.0
TabICL ICL-mean	7	7	7	3.5	3.5	3.5
PyMFE	5	5	2	4.0	4.0	3.5
Frozen TabPFN	2	3	3	5.0	5.0	5.0
Dataset2Vec	4	6	5	5.0	5.5	5.0

Appendix

Table 15: Sensitivity of the six-task aggregate ranking to the TabRepo benchmark-selection budget. Only the TabRepo K-means evaluation changes across columns; the other five evaluation units are held fixed. The aggregate score is the median rank across the six evaluation units.

Task	Dataset ID	Gap	ID rows	OOD rows
ASSISTments	assistments	0.3519	8192	1906
FICO HELOC	heloc	0.2826	278	6914
College Scorecard	college_scorecard	0.1132	8192	1352
Hypertension	brfss_blood_pressure	0.0816	8192	8192
Hospital Readmission	diabetes_readmission	0.0489	4287	8192
Diabetes	brfss_diabetes	0.0450	8192	8192

Appendix

Table 16: TableShift tasks used for the shift-severity evaluation. The accuracy gap is the mean ID-minus-OOD accuracy difference across estimators with both published ID and OOD results. Row counts are the actual ID and OOD samples embedded after the at-most-8,192-row sampling described in Section 4 .

Figure 4: Encoder time and memory scaling as the number of rows and features increases. For the row-scaling experiment, the number of features is fixed at

d=64

; for the feature-scaling experiment, the number of observations is fixed at

n=10{,}000

. TabMF -ICL closely tracks TabICL ICL-mean, while TabMF -PFN closely tracks TabPFN mean-pooling, indicating that the dataset-level readouts introduce little additional inference cost relative to their respective backbones. HyTrel exceeds available memory at larger table sizes. Neural Statistician and Dataset2Vec use bounded input sampling, which limits their scaling with table size.

Encoder	MAP $\uparrow$	Shuffling $\uparrow$	Type F1 $\uparrow$
TabMF -ICL	$0.605\pm 0.158$	$0.732\pm 0.281$	0.363
TabMF -PFN	$0.448\pm 0.117$	$0.773\pm 0.251$	0.289
TabICL ICL-mean	$0.468\pm 0.137$	$0.753\pm 0.266$	—
HyTrel	$0.684\pm 0.146$	$0.977\pm 0.053$	0.681
PyMFE	$0.456\pm 0.137^{\dagger}$	$0.705\pm 0.150$	0.372
Encoder	HPO quality $\uparrow$	HPO regret@5 $\downarrow$	Targets

Appendix

Table 17: Supplementary transfer evaluations. TEmBed MAP covers 16 datalakes except PyMFE (†: 17); shuffling covers 60 slices. Standard deviations are across datalakes or slices. Similarity and shuffling use uncapped inputs. HPO covers 82 targets except HyTrel (‡: 77). A dash denotes an unavailable result.

Panel of Figure 5 — Figure 5: The two TabMF encoders. TabMF -ICL (left) appends a learned summary token to the target-conditioned row states and reads its state after the pretrained ICL Transformer and final normalization. TabMF -PFN (right) pools the backbone’s feature-group states over labeled observations, models their interactions with a learned column Transformer, and averages the active groups. Each readout feeds a projection network; gray blocks are pretrained and fine-tuned, while green blocks are added and trained from scratch.

Explore similar work

Tabular foundation models for non-tabular tasks
Aug 23, 2026Goran Nakerst, John Brennan, Wouter Beugeling +1Tabular Foundation Models Cross-Domain Generalization
Institut für Theoretische Physik, Technische Universität Dresden, 01062 Dresden, Germany · Department of Physics, Maynooth University, Ireland · Physikalisches Institut (EP3) and Institute for Topological Insulators, Universität Würzburg, Am Hubland, 97074 Würzburg, Germany
TabFM: A Zero-Shot Foundation Model for Tabular Data
Sep 29, 2026Weihao Kong, Erez Louidor Ilan, Shuxin Nie +6Tabular Foundation Models Tabular Prediction
Google Research
Understanding the Surprising Generalization Properties of Tabular Foundation Models
Aug 18, 2026Nour Shaheen, Junwei Ma, Alex Labach +3Tabular Foundation Models Tabular ICL
Polytechnique Montréal · Mila – Quebec AI Institute · Chandar Research Lab +6

Aug 23, 2026, cs.LGJ/K move · Enter open · S save

Tabular foundation models for non-tabular tasks

Goran Nakerst, John Brennan, Wouter Beugeling, Masudul Haque

Institut für Theoretische Physik, Technische Universität Dresden, 01062 Dresden, Germany · Department of Physics, Maynooth University, Ireland · Physikalisches Institut (EP3) and Institute for Topological Insulators, Universität Würzburg, Am Hubland, 97074 Würzburg, Germany

Tabular Foundation Models Cross-Domain Generalization Tabular ICL

Open paper AskPDF arXiv

Tabular foundation models (TFMs) have recently emerged as a promising paradigm for machine learning on tabular data, offering the ability to generalize across datasets without task-specific training. Since many machine learning datasets can be represented as tables, this raises the question: does TFM capability extend beyond tasks traditionally regarded as tabular? We address this question by using TabPFN v3 on three non-tabular classification problems: handwritten digit recognition on MNIST, language identification of French and German words, and image classification on Tiny ImageNet. In each case, the original data are represented as rows of a table and classification is formulated as prediction of a missing label. We evaluate performance as a function of the number of context samples provided to the pretrained model, with no additional training or fine-tuning. Despite having no explicit access to the spatial or sequential structure characterizing the data, TabPFN v3 in some cases achieves accuracies comparable with that of models or methods geared specifically toward the corresponding tasks.

Abstract

Explore similar work

Tabular foundation models for non-tabular tasks

TabFM: A Zero-Shot Foundation Model for Tabular Data

Understanding the Surprising Generalization Properties of Tabular Foundation Models