cs.LGAug 26, 2026

A Comparative Framework for Evaluating Foundation Models on Tabular Data: A Case Study in Healthcare

Authors: Majid Lotfian Delouee, Sjors G. J. G. In 't Veld, Martijn C. Schut

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Tabular data is the most common format in clinical practice, encompassing laboratory results, medication records, diagnostic codes, and patient demographics. As foundation models for tabular data have grown in number and variety, a practical question has become harder to answer: which model should a clinician or data scientist actually choose for a given task, and why? Existing surveys catalogue what these models can do, but they stop short of providing a structured way to compare them against the specific demands of a real application. We introduce \system{}, a comparative evaluation framework that scores and ranks tabular foundation models (TFMs) across six clinically meaningful dimensions: how well a model generalizes to new datasets, how effectively it protects patient privacy, how much data it needs to perform well, how it scales with growing datasets and feature spaces, how interpretable its predictions are to clinicians, and how fairly it performs across patient subgroups. Each dimension is broken down into measurable sub-components, and groups of sub-components can optionally be combined into supplementary compound scores, called super-metrics, that provide a diagnostic view of how a model performs across several dimensions simultaneously. To show how the framework works in practice, we apply it to two healthcare use cases, screening for iron deficiency and predicting heart failure, demonstrating how the same set of metrics leads to different model rankings depending on what matters most in each clinical context. We also provide a taxonomy of 45 TFMs organized by their underlying architecture, which serves as a reference for researchers and practitioners looking to navigate this rapidly expanding field.

Figures & tables

Figure 1: The three-layer evaluation architecture of OpTFM . Sub-metrics (bottom, grouped by metric domain) are atomic measurable properties derived from documented model design characteristics. Metrics (middle) aggregate sub-metrics within six clinical evaluation dimensions using domain-specific weights. Super-metrics (top) are optional supplementary compound scores that combine sub-metrics across domain boundaries to provide additional diagnostic depth. The overall ranking R\mathcal{R} integrates all sub-metric scores into a single recommended model for the target deployment context.
Figure 2: Sub-metric composition of the four supplementary super-metrics, shown as radar plots at reference maximum values. TLC draws from G, P, and D; MR from G, P, and D; EPT from P, D, and S; CT from I and F. Together they cover all six metric domains. When actual model scores replace the reference values, the polygon shape makes cross-domain strengths and deficits visible as a diagnostic complement to the overall ranking R\mathcal{R} .
PropertyBrief DescriptionSub-metrics Decreased by 1 (if absent)
Built-in Pre-TrainingThe model has a genuine pre-training phase (self-supervised pretraining on auxiliary data, or a meta-learned/prior-fitted setup used before task-specific application), increasing its adaptability and performance in low-data and zero-shot scenarios. End-to-end supervised training on the target dataset alone does not satisfy this property.IDT, CDT, FAC, ZIA, PRD, FLC
On-Premises Data ProcessingThe model processes all data locally without sending raw or sensitive records to external services, preserving privacy and regulatory compliance.DAR, MIR, DPC, DRAR, TPDHC
Handle Missing DataThe model can gracefully handle incomplete records, increasing robustness to schema and noise variations.SR, DNR
Explainability ToolsThe model provides interpretable outputs or feature importance, supporting transparency and trust.FIC, MTS, EC, LI
Predictive Fairness ConstraintsThe model incorporates mechanisms to equalize positive predictive value across demographic groups, reducing disparate impact in high-risk predictions.PPBS, IFS
Bias Amplification ControlThe model does not amplify biases present in the training data; its predictions are no more biased across groups than the training data itself.BAS
Table 1: Properties for Tabular Foundation Models. The absence of each property decreases the corresponding sub-metrics by 1, starting from a default value of 10. Built-in Pre-Training and Data Anonymization Techniques each now govern one additional sub-metric (ZIA and TPDHC respectively) relative to the original mapping, correcting two gaps under which those sub-metrics could not be reduced below the maximum for any model regardless of documented capability.
Figure 3: Sub-metric scores for 16 tabular foundation models across all 25 sub-metrics, derived from documented design properties starting from a default of 10. Columns are grouped by metric domain (G = Generalizability, P = Privacy, D = Data Efficiency, S = Scalability, I = Interpretability, F = Fairness). Green cells indicate maximum scores; yellow and orange indicate single- and double-point deductions; red indicates three or more points below the maximum. Models are ordered from highest to lowest aggregate score. The full property-to-sub-metric mapping is given in Table 1 .
Figure 4: Metric-layer performance profile under Iron Deficiency use-case weights, computed from the independent-literature-based property audit (Section 5.4.1 ). Each cell shows the raw metric score relative to the use-case-adjusted maximum and the corresponding percentage. Blue borders mark the highest-scoring model in each metric column. The eight models shown are the eight with the strongest combined rank across both use cases (Section 5.3 ); no model is pre-selected to illustrate a trade-off. R\mathcal{R} is the overall weighted ranking score (max = 1020).
ModelG\mathcal{G}P\mathcal{P}D\mathcal{D}S\mathcal{S}I\mathcal{I}F\mathcal{F}R(τi)\mathcal{R}(\tau_{i})
Max possible1902101801101901401020
TabPFN 2.5190189180110171126966
TabICL190189180110171126966
MediTab19021018085171126962
TAPTAP19021018081171126958
VIME186189180106171126958
Table 2: Model ranking for the iron deficiency use case, computed from the audited property assignments (Section 5.4 ). Scores represent the weighted sum R(τi)\mathcal{R}(\tau_{i}) across all sub-metrics under the iron deficiency sub-metric weights. The maximum possible score is 1020. The eight models shown are the eight with the strongest combined rank across both use cases (Section 5.3 ); no model is pre-selected to illustrate a trade-off.
ModelTLC (340)MR (170)EPT (500)CT (330)
TabPFN 2.5333166479297
TabICL333166479297
MediTab340170475297
TAPTAP340170471297
VIME329162475297
SAINT325158446316
Table 3: Super-metric scores for the iron deficiency use case. TLC = Transfer Learning Capacity ( TLC\mathcal{TLC} , max 340), MR = Model Resilience ( MR\mathcal{MR} , max 170), EPT = Efficiency-Privacy Trade-off ( EPT\mathcal{EPT} , max 500), CT = Clinical Trustworthiness ( CT\mathcal{CT} , max 330). Scores are weighted sums using the iron deficiency sub-metric weights.
Figure 5: Metric-layer performance profile under Heart Failure use-case weights, computed from the same independent-literature-based property audit (Section 5.4.1 ). Each cell shows the raw metric score relative to the use-case-adjusted maximum and the corresponding percentage. Blue borders mark the highest-scoring model in each metric column. The same eight models appear as in Figure 4 ; comparing the two figures shows the ranking reorders within the group rather than a fixed top group holding constant. R\mathcal{R} is the overall weighted ranking score (max = 1070).
ModelG\mathcal{G}P\mathcal{P}D\mathcal{D}S\mathcal{S}I\mathcal{I}F\mathcal{F}R(τi)\mathcal{R}(\tau_{i})
Max possible2002201501601901501070
TabPFN 2.52001981501601711351014
TabICL2001981501601711351014
VIME1951981501551711351004
MediTab200220150122171135998
TAPTAP200220150117171135993
Table 4: Model ranking for the heart failure use case, computed from the audited property assignments (Section 5.4 ). Scores represent the weighted sum R(τi)\mathcal{R}(\tau_{i}) across all sub-metrics under the heart failure sub-metric weights. The maximum possible score is 1070. The same eight models appear as in Table 2 ; comparing the two tables shows the ranking reorders within the group – most notably, SAINT falls from sixth in the iron deficiency table to last here, as the heavier heart failure scalability weighting exposes its independently-documented resource-utilization weakness, while TabDPT and XTab, whose scalability scores were not revised by this audit, close the gap and pass it – rather than a fixed top group holding constant regardless of clinical priority.
ModelTLC (370)MR (190)EPT (530)CT (340)
TabPFN 2.5361185508306
TabICL361185508306
VIME356180503306
MediTab370190492306
TAPTAP370190487306
TabDPT341165493306
Table 5: Super-metric scores for the heart failure use case. TLC = Transfer Learning Capacity ( TLC\mathcal{TLC} , max 370), MR = Model Resilience ( MR\mathcal{MR} , max 190), EPT = Efficiency-Privacy Trade-off ( EPT\mathcal{EPT} , max 530), CT = Clinical Trustworthiness ( CT\mathcal{CT} , max 340). Scores are weighted sums using the heart failure sub-metric weights.
More in this topic

Tabular Foundation Models

Tabular foundation models learn reusable representations or in-context prediction skills for structured data such as spreadsheets and databases.

See all 187 papers

Explore similar work

CardsList
Jun 29, 2026

Beyond IID: How General Are Tabular Foundation Models, Really?

Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluating tabular foundation models on diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks where tabular foundation models already excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements on IID data rather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unified benchmarking beyond standard benchmarks, we introduce Data Foundry, a Python framework and metadata schema for curating tabular datasets for predictive machine learning. Our results across 11 models and 142 curated datasets show that existing tabular foundation models excel on tiny- to medium-sized IID data, while traditional tree-based and deep learning models still dominate on non-IID, large, and high-dimensional datasets. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.

Lennart Purucker, Andrej Tschalzev, Nick Erickson +7

1Prior Labs · University of Mannheim · 4INRIA Saclay +5

Tabular Foundation ModelsOOD GeneralizationTabular MLTabular PredictionCross-Dataset Generalization
AskPDF
May 27, 2026

Explaining Tabular Foundation Model Differences Through Meta-Features

With the rise of tabular foundation models alongside traditional models still performing well on many tasks, choosing the right model for a tabular dataset remains difficult. We investigate whether dataset meta-features can explain performance gaps between model families on tabular prediction tasks. Using the TabArena benchmark results, we analyze dataset-level performance gaps and relate them to model-agnostic meta-features. After strict statistical tests with false discovery control, we find that (1) for neural network vs. tree gaps, no meta-feature survives false discovery control, (2) for non-foundation vs. foundation model gaps, one association is robust but does not generalize when tested in leave-one-dataset-out prediction, and (3) for TabICLv2 vs. TabPFN-2.6, one robust association also improves held-out prediction. Furthermore, we conduct a leave-one-dataset-out analysis and find that meta-feature predictors fail to improve meaningfully over a simple baseline. Overall, our results show the heterogeneity of tabular datasets and that global meta-feature approaches are not robust enough to offer explanations on the 51 TabArena datasets.

Markus Herre, Andrej Tschalzev, Sascha Marton +1

Clausthal University of Technology, Clausthal-Zellerfeld, Germany · University of Mannheim, Mannheim, Germany

Tabular Foundation ModelsTabular MLTabular PredictionCross-Dataset Generalization
AskPDF
Jun 28, 2026

Towards Evaluating Data Priors for Tabular Foundation Models

Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However, priors are rarely evaluated as independent components, making it difficult to understand how much they affect downstream model behavior. This raises a methodological question: how can priors from different tabular foundation models be compared independently of the architectures and training protocols they were introduced with? To study this question, we implement a unified interface for publicly available priors from recent tabular foundation models and priors constructed from real datasets. We generate training tasks from each prior, train the same model architecture under a fixed training protocol, and evaluate the resulting models on shared downstream classification tasks. We compare priors through both generated-task statistics and downstream predictive performance. Our results show that different priors favor different downstream behaviors, with some achieving stronger absolute performance and others exhibiting more consistent relative rankings across datasets. We further find that data-level similarity only partially explains downstream behavior. Our code is available at https://github.com/automl/TFM-Playground/tree/prior-dev.

Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle +1

Department of Computer Science, University of Freiburg, Freiburg, Germany · Zuse School ELIZA, Hochschulstr. 10, 64289 Darmstadt · Prior Labs, Freiburg, Germany +1

Tabular Foundation ModelsSynthetic Data Evaluation
AskPDF