A Comparative Framework for Evaluating Foundation Models on Tabular Data: A Case Study in Healthcare

Majid Lotfian Delouee; Sjors G. J. G. In 't Veld; Martijn C. Schut

cs.LGAug 26, 2026

A Comparative Framework for Evaluating Foundation Models on Tabular Data: A Case Study in Healthcare

Authors: Majid Lotfian Delouee, Sjors G. J. G. In 't Veld, Martijn C. Schut

Abstract

Tabular data is the most common format in clinical practice, encompassing laboratory results, medication records, diagnostic codes, and patient demographics. As foundation models for tabular data have grown in number and variety, a practical question has become harder to answer: which model should a clinician or data scientist actually choose for a given task, and why? Existing surveys catalogue what these models can do, but they stop short of providing a structured way to compare them against the specific demands of a real application. We introduce \system{}, a comparative evaluation framework that scores and ranks tabular foundation models (TFMs) across six clinically meaningful dimensions: how well a model generalizes to new datasets, how effectively it protects patient privacy, how much data it needs to perform well, how it scales with growing datasets and feature spaces, how interpretable its predictions are to clinicians, and how fairly it performs across patient subgroups. Each dimension is broken down into measurable sub-components, and groups of sub-components can optionally be combined into supplementary compound scores, called super-metrics, that provide a diagnostic view of how a model performs across several dimensions simultaneously. To show how the framework works in practice, we apply it to two healthcare use cases, screening for iron deficiency and predicting heart failure, demonstrating how the same set of metrics leads to different model rankings depending on what matters most in each clinical context. We also provide a taxonomy of 45 TFMs organized by their underlying architecture, which serves as a reference for researchers and practitioners looking to navigate this rapidly expanding field.

Figures & tables

Figure 1: The three-layer evaluation architecture of OpTFM . Sub-metrics (bottom, grouped by metric domain) are atomic measurable properties derived from documented model design characteristics. Metrics (middle) aggregate sub-metrics within six clinical evaluation dimensions using domain-specific weights. Super-metrics (top) are optional supplementary compound scores that combine sub-metrics across domain boundaries to provide additional diagnostic depth. The overall ranking $\mathcal{R}$ integrates all sub-metric scores into a single recommended model for the target deployment context.

Panel of Figure 2 — Figure 2: Sub-metric composition of the four supplementary super-metrics, shown as radar plots at reference maximum values. TLC draws from G, P, and D; MR from G, P, and D; EPT from P, D, and S; CT from I and F. Together they cover all six metric domains. When actual model scores replace the reference values, the polygon shape makes cross-domain strengths and deficits visible as a diagnostic complement to the overall ranking $\mathcal{R}$ .

Property	Brief Description	Sub-metrics Decreased by 1 (if absent)
Built-in Pre-Training	The model has a genuine pre-training phase (self-supervised pretraining on auxiliary data, or a meta-learned/prior-fitted setup used before task-specific application), increasing its adaptability and performance in low-data and zero-shot scenarios. End-to-end supervised training on the target dataset alone does not satisfy this property.	IDT, CDT, FAC, ZIA, PRD, FLC
On-Premises Data Processing	The model processes all data locally without sending raw or sensitive records to external services, preserving privacy and regulatory compliance.	DAR, MIR, DPC, DRAR, TPDHC
Handle Missing Data	The model can gracefully handle incomplete records, increasing robustness to schema and noise variations.	SR, DNR
Explainability Tools	The model provides interpretable outputs or feature importance, supporting transparency and trust.	FIC, MTS, EC, LI
Predictive Fairness Constraints	The model incorporates mechanisms to equalize positive predictive value across demographic groups, reducing disparate impact in high-risk predictions.	PPBS, IFS
Bias Amplification Control	The model does not amplify biases present in the training data; its predictions are no more biased across groups than the training data itself.	BAS

Table 1: Properties for Tabular Foundation Models. The absence of each property decreases the corresponding sub-metrics by 1, starting from a default value of 10. Built-in Pre-Training and Data Anonymization Techniques each now govern one additional sub-metric (ZIA and TPDHC respectively) relative to the original mapping, correcting two gaps under which those sub-metrics could not be reduced below the maximum for any model regardless of documented capability.

Figure 3: Sub-metric scores for 16 tabular foundation models across all 25 sub-metrics, derived from documented design properties starting from a default of 10. Columns are grouped by metric domain (G = Generalizability, P = Privacy, D = Data Efficiency, S = Scalability, I = Interpretability, F = Fairness). Green cells indicate maximum scores; yellow and orange indicate single- and double-point deductions; red indicates three or more points below the maximum. Models are ordered from highest to lowest aggregate score. The full property-to-sub-metric mapping is given in Table 1 .

Panel of Figure 4 — Figure 4: Metric-layer performance profile under Iron Deficiency use-case weights, computed from the independent-literature-based property audit (Section 5.4.1 ). Each cell shows the raw metric score relative to the use-case-adjusted maximum and the corresponding percentage. Blue borders mark the highest-scoring model in each metric column. The eight models shown are the eight with the strongest combined rank across both use cases (Section 5.3 ); no model is pre-selected to illustrate a trade-off. $\mathcal{R}$ is the overall weighted ranking score (max = 1020).

Model	$\mathcal{G}$	$\mathcal{P}$	$\mathcal{D}$	$\mathcal{S}$	$\mathcal{I}$	$\mathcal{F}$	$\mathcal{R}(\tau_{i})$
Max possible	190	210	180	110	190	140	1020
TabPFN 2.5	190	189	180	110	171	126	966
TabICL	190	189	180	110	171	126	966
MediTab	190	210	180	85	171	126	962
TAPTAP	190	210	180	81	171	126	958
VIME	186	189	180	106	171	126	958

Table 2: Model ranking for the iron deficiency use case, computed from the audited property assignments (Section 5.4 ). Scores represent the weighted sum

\mathcal{R}(\tau_{i})

across all sub-metrics under the iron deficiency sub-metric weights. The maximum possible score is 1020. The eight models shown are the eight with the strongest combined rank across both use cases (Section 5.3 ); no model is pre-selected to illustrate a trade-off.

Model	TLC (340)	MR (170)	EPT (500)	CT (330)
TabPFN 2.5	333	166	479	297
TabICL	333	166	479	297
MediTab	340	170	475	297
TAPTAP	340	170	471	297
VIME	329	162	475	297
SAINT	325	158	446	316

Table 3: Super-metric scores for the iron deficiency use case. TLC = Transfer Learning Capacity (

\mathcal{TLC}

, max 340), MR = Model Resilience (

\mathcal{MR}

, max 170), EPT = Efficiency-Privacy Trade-off (

\mathcal{EPT}

, max 500), CT = Clinical Trustworthiness (

\mathcal{CT}

, max 330). Scores are weighted sums using the iron deficiency sub-metric weights.

Panel of Figure 5 — Figure 5: Metric-layer performance profile under Heart Failure use-case weights, computed from the same independent-literature-based property audit (Section 5.4.1 ). Each cell shows the raw metric score relative to the use-case-adjusted maximum and the corresponding percentage. Blue borders mark the highest-scoring model in each metric column. The same eight models appear as in Figure 4 ; comparing the two figures shows the ranking reorders within the group rather than a fixed top group holding constant. $\mathcal{R}$ is the overall weighted ranking score (max = 1070).

Model	$\mathcal{G}$	$\mathcal{P}$	$\mathcal{D}$	$\mathcal{S}$	$\mathcal{I}$	$\mathcal{F}$	$\mathcal{R}(\tau_{i})$
Max possible	200	220	150	160	190	150	1070
TabPFN 2.5	200	198	150	160	171	135	1014
TabICL	200	198	150	160	171	135	1014
VIME	195	198	150	155	171	135	1004
MediTab	200	220	150	122	171	135	998
TAPTAP	200	220	150	117	171	135	993

Table 4: Model ranking for the heart failure use case, computed from the audited property assignments (Section 5.4 ). Scores represent the weighted sum

\mathcal{R}(\tau_{i})

across all sub-metrics under the heart failure sub-metric weights. The maximum possible score is 1070. The same eight models appear as in Table 2 ; comparing the two tables shows the ranking reorders within the group – most notably, SAINT falls from sixth in the iron deficiency table to last here, as the heavier heart failure scalability weighting exposes its independently-documented resource-utilization weakness, while TabDPT and XTab, whose scalability scores were not revised by this audit, close the gap and pass it – rather than a fixed top group holding constant regardless of clinical priority.

Model	TLC (370)	MR (190)	EPT (530)	CT (340)
TabPFN 2.5	361	185	508	306
TabICL	361	185	508	306
VIME	356	180	503	306
MediTab	370	190	492	306
TAPTAP	370	190	487	306
TabDPT	341	165	493	306

Table 5: Super-metric scores for the heart failure use case. TLC = Transfer Learning Capacity (

\mathcal{TLC}

, max 370), MR = Model Resilience (

\mathcal{MR}

, max 190), EPT = Efficiency-Privacy Trade-off (

\mathcal{EPT}

, max 530), CT = Clinical Trustworthiness (

\mathcal{CT}

, max 340). Scores are weighted sums using the heart failure sub-metric weights.

Tabular Foundation Models

Tabular foundation models learn reusable representations or in-context prediction skills for structured data such as spreadsheets and databases.

See all 187 papers

Beyond IID: How General Are Tabular Foundation Models, Really?
Jun 29, 2026Lennart Purucker, Andrej Tschalzev, Nick Erickson +7Tabular Foundation Models OOD Generalization
1Prior Labs · University of Mannheim · 4INRIA Saclay +5
Explaining Tabular Foundation Model Differences Through Meta-Features
May 27, 2026Markus Herre, Andrej Tschalzev, Sascha Marton +1Tabular Foundation Models Tabular ML
Clausthal University of Technology, Clausthal-Zellerfeld, Germany · University of Mannheim, Mannheim, Germany
Towards Evaluating Data Priors for Tabular Foundation Models
Jun 28, 2026Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle +1Tabular Foundation Models Synthetic Data Evaluation
Department of Computer Science, University of Freiburg, Freiburg, Germany · Zuse School ELIZA, Hochschulstr. 10, 64289 Darmstadt · Prior Labs, Freiburg, Germany +1

Jun 29, 2026J/K move · Enter open · S save

Beyond IID: How General Are Tabular Foundation Models, Really?

Lennart Purucker, Andrej Tschalzev, Nick Erickson, Gioia Blayer+6

1Prior Labs · University of Mannheim · 4INRIA Saclay+5

Tabular Foundation Models OOD Generalization Tabular ML Tabular Prediction Cross-Dataset Generalization

Open paper AskPDF arXiv

Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluating tabular foundation models on diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks where tabular foundation models already excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements on IID data rather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unified benchmarking beyond standard benchmarks, we introduce Data Foundry, a Python framework and metadata schema for curating tabular datasets for predictive machine learning. Our results across 11 models and 142 curated datasets show that existing tabular foundation models excel on tiny- to medium-sized IID data, while traditional tree-based and deep learning models still dominate on non-IID, large, and high-dimensional datasets. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.

Abstract

Tabular Foundation Models

Explore similar work

Beyond IID: How General Are Tabular Foundation Models, Really?

Explaining Tabular Foundation Model Differences Through Meta-Features

Towards Evaluating Data Priors for Tabular Foundation Models